A training method of a retrieval model and a related device

Abstract

Embodiments of the present application disclose a training method of a retrieval model and a related device, which can be applied to a vehicle-mounted scene. A first vector of first content and a second vector of second content in a sample pair are obtained. The first vector and the second vector include multiple channel features, each channel feature representing different information. The first vector and the second vector are input into an initial retrieval model, and through the initial retrieval model, multiple undetermined similarities between the first vector and the second vector are obtained according to the association between each first channel feature and at least two second channel features. The number of interactions between the multiple channel features included in different vectors is increased, and the information exchange between the first vector and the second vector is strengthened. The maximum undetermined similarity in the multiple undetermined similarities is taken as the similarity between the first vector and the second vector, so that the parameters of the initial model are trained through more accurate similarities and sample labels, and the accuracy of the obtained retrieval model is higher.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method for training a retrieval model and related apparatus. Background Technology

[0002] With the development of the internet, people can access all kinds of information anytime. For example, when a user wants to learn about a certain content, they can enter the relevant keywords into a search engine. The search engine will then retrieve content that matches the keywords from its database and display it to the user.

[0003] In related technologies, the similarity between keywords and the content to be displayed is calculated. If the similarity exceeds a preset threshold, the content to be displayed is shown to the user as the content that matches the keywords. However, this method has a low accuracy rate. Summary of the Invention

[0004] To address the aforementioned technical problems, this application provides a method and apparatus for training a retrieval model, which improves the accuracy of the retrieval model.

[0005] The embodiments of this application disclose the following technical solutions:

[0006] On one hand, embodiments of this application provide a method for training a retrieval model, the method comprising:

[0007] Obtain sample pairs with sample tags, wherein the sample pairs include first content and second content, and the sample tags are used to identify whether the first content and the second content are related;

[0008] Obtain a first vector for identifying the first content and a second vector for identifying the second content; the first vector includes N first channel features and the second vector includes M second channel features, where N is an integer greater than 1 and M is an integer greater than 1.

[0009] The first vector and the second vector are input into the initial retrieval model. The initial retrieval model obtains L undetermined similarities between the first vector and the second vector based on the correlation between each first channel feature and at least two second channel features; L is an integer greater than 1.

[0010] The largest of the L undetermined similarities is taken as the similarity between the first vector and the second vector.

[0011] The retrieval model is obtained by training the parameters of the initial retrieval model based on the similarity between the first vector and the second vector, and the sample labels.

[0012] On the other hand, embodiments of this application provide a training apparatus for a retrieval model, the apparatus comprising: a first acquisition unit, a second acquisition unit, a first determination unit, a second determination unit, and a training unit;

[0013] The first acquisition unit is used to acquire sample pairs with sample tags, the sample pairs including first content and second content, and the sample tags are used to identify whether the first content and the second content are related;

[0014] The second acquisition unit is used to acquire a first vector for identifying the first content and a second vector for identifying the second content; the first vector includes N first channel features and the second vector includes M second channel features, where N is an integer greater than 1 and M is an integer greater than 1.

[0015] The first determining unit is used to input the first vector and the second vector into an initial retrieval model, and obtain L undetermined similarities between the first vector and the second vector based on the correlation between each of the first channel features and at least two of the second channel features; where L is an integer greater than 1.

[0016] The second determining unit is used to take the largest undetermined similarity among the L undetermined similarities as the similarity between the first vector and the second vector;

[0017] The training unit is used to train the parameters of the initial retrieval model based on the similarity between the first vector and the second vector, and the sample labels, to obtain the retrieval model.

[0018] On the other hand, embodiments of this application provide a computer device, the computer device including a processor and a memory:

[0019] The memory is used to store computer programs and to transfer the computer programs to the processor;

[0020] The processor is configured to execute the methods described above according to instructions in the computer program.

[0021] On the other hand, embodiments of this application provide a computer-readable storage medium for storing a computer program for performing the methods described above.

[0022] On the other hand, embodiments of this application provide a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods described above.

[0023] As can be seen from the above technical solution, a sample pair consisting of first content and second content is obtained, and this sample pair has sample labels that identify whether the first content and the second content are related. A first vector for identifying the first content and a second vector for identifying the second content are obtained. The first vector includes multiple first channel features, and the second vector includes multiple second channel features. Since the information represented by each channel feature is different, in order to allow the first vector and the second vector to interact better, the first vector and the second vector are input into an initial retrieval model. The initial retrieval model obtains multiple undetermined similarities between the first vector and the second vector based on the correlation between each first channel feature and at least two second channel features, increasing the number of interactions between the multiple channel features included in different vectors and strengthening the information exchange between the first vector and the second vector. Using the largest undetermined similarity among multiple undetermined similarities as the similarity between the first vector and the second vector is more accurate than the similarity obtained through only one vector interaction. Therefore, by training the parameters of the initial model with more accurate similarities and sample labels, the accuracy of the obtained retrieval model is higher. Therefore, by allowing multiple channel features between different vectors to interact, the number of interactions between channel features between vectors is increased, thereby strengthening the information exchange between different vectors. This makes the similarity between different vectors more accurate, and the retrieval model trained based on the more accurate similarity is more accurate and has better generalization performance. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A schematic diagram illustrating an application scenario of the training method for the retrieval model provided in this embodiment;

[0026] Figure 2 A flowchart illustrating the training method of the retrieval model provided in this application embodiment;

[0027] Figure 3This is a schematic diagram illustrating a method for determining the similarity of a target object using a rolling mechanism, as provided in an embodiment of this application.

[0028] Figure 4 This is a schematic diagram illustrating how to determine the undetermined similarity through full-channel multiplication, as provided in an embodiment of this application.

[0029] Figure 5 A schematic diagram illustrating a scenario for training and applying a retrieval model, as provided in an embodiment of this application.

[0030] Figure 6 A schematic diagram of the structure of a training device for a retrieval model provided in an embodiment of this application;

[0031] Figure 7 This is a schematic diagram of the server structure provided in an embodiment of this application;

[0032] Figure 8 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Detailed Implementation

[0033] The embodiments of this application will now be described with reference to the accompanying drawings.

[0034] In related technologies, the similarity between keywords and content to be displayed is calculated by multiplying by position. For example, the keyword vector corresponding to the keyword includes three channel features, and the content vector corresponding to the content to be displayed includes three channel features. The channel features between different vectors are multiplied by position. For example, the channel feature in the first position of the keyword vector is multiplied by the channel feature in the first position of the content vector to be displayed, and so on, so as to obtain the similarity between the keyword vector and the content vector to be displayed.

[0035] However, because different channel features represent different information, positional multiplication only allows interaction between channel features at corresponding positions; different vectors cannot interact effectively, resulting in low accuracy in calculating similarity between different vectors. For example, the information that best represents the meaning of a keyword may be located in the second channel feature of the keyword vector, while the information that best represents the meaning of the content to be displayed may be located in the third channel feature of the content vector. Positional multiplication cannot allow these two channel features to interact, potentially leading to the conclusion that two otherwise similar vectors are dissimilar. In other words, positional multiplication yields low accuracy.

[0036] Based on this, embodiments of this application provide a training method and related apparatus for a retrieval model. By allowing multiple channel features between different vectors to interact, the number of interactions between the channel features of the vectors is increased, thereby strengthening the information exchange between different vectors and making the similarity between different vectors more accurate. As a result, the retrieval model trained based on the more accurate similarity has higher accuracy and better generalization performance.

[0037] The training method for the retrieval model provided in this application is based on artificial intelligence (AI). AI is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. In other words, AI is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to have the functions of perception, reasoning, and decision-making.

[0038] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0039] In the embodiments of this application, the main artificial intelligence technologies involved include the aforementioned machine learning / deep learning and other directions.

[0040] The training method for the retrieval model provided in this application can be applied to training devices with data processing capabilities, such as terminal devices and servers. Terminal devices include, but are not limited to, mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle terminals, and aircraft; servers can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing cloud computing services. Terminal devices and servers can be directly or indirectly connected via wired or wireless communication, and this application does not impose any restrictions on this connection.

[0041] This training device also possesses machine learning capabilities. Machine learning is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, and many other disciplines. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks.

[0042] In the training method of the retrieval model provided in the embodiments of this application, the artificial intelligence model used mainly involves the application of machine learning, and the retrieval model is trained through machine learning and other methods.

[0043] To facilitate understanding of the training method of the retrieval model provided in this application embodiment, the following example uses a server as the execution subject of the training method of the retrieval model to illustrate the application scenario of the training method of the retrieval model.

[0044] See Figure 1 This figure is a schematic diagram illustrating an application scenario of the training method for the retrieval model provided in an embodiment of this application. For example... Figure 1 As shown, this application scenario includes a server 110, which is used to execute the training method of the retrieval model provided in this application embodiment. It should be noted that the first content and the second content are different. The following explanation uses the example of the first content being text and the second content being video.

[0045] Server 110 retrieves a sample pair consisting of text A and video B, and this sample pair has sample tags used to identify whether the first and second content are related. If text A and video B are related, then inputting text A will return search results for video B. For example, if text A is "videos about cars," then video B will be some videos related to cars.

[0046] Server 110 retrieves the text vector (first vector) of text A and the video vector (second vector) of video B. The text vector includes four first-channel features, and the video vector includes four second-channel vectors.

[0047] Since the information represented by each channel feature is different, in order to enable better interaction between the first vector and the second vector, the server 110 inputs the text vector and video vector into the initial retrieval model. The initial retrieval model obtains two undetermined similarities between the first vector and the second vector based on the correlation between each first channel feature and the two second channel features. Compared with the positional multiplication method, this increases the number of interactions between the multiple channel features included in different vectors (from 1 time to 2 times), and strengthens the information exchange between the first vector and the second vector.

[0048] Server 110 uses the largest of multiple undetermined similarities as the similarity between the text vector and the video vector, which is more accurate than the similarity obtained through only one vector interaction. Therefore, by training the parameters of the initial model with a more accurate similarity and sample labels, the resulting retrieval model has higher accuracy.

[0049] Therefore, by allowing multiple channel features between different vectors to interact, the number of interactions between channel features between vectors is increased, thereby strengthening the information exchange between different vectors. This makes the similarity between different vectors more accurate, and the retrieval model trained based on the more accurate similarity is more accurate and has better generalization performance.

[0050] The training method for the retrieval model provided in this application embodiment can be executed by a server. However, in other embodiments of this application, the terminal device may also have similar functions to the server to execute the training method for the retrieval model provided in this application embodiment, or the terminal device and the server may jointly execute the training method for the retrieval model provided in this application embodiment. This embodiment does not limit this.

[0051] The training method of the retrieval model provided in this application will be described in detail below through method embodiments.

[0052] See Figure 2 This figure is a schematic flowchart illustrating the training method of the retrieval model provided in an embodiment of this application. For ease of description, the following embodiments will still use a server as the execution entity of the retrieval model training method as an example. Figure 2 As shown, the training method for this retrieval model includes the following steps:

[0053] S201: Obtain sample pairs with sample labels.

[0054] In this application, each sample pair consists of a first content and a second content, which are distinct from each other. The first and second content can be different content of the same data type, or they can be different content of different data types; this application does not impose specific limitations in this regard. The first and second content can be content of data types such as video, text, and audio. For example, the first content can be text content, and the second content can be audio content.

[0055] Each sample pair has a sample label, which identifies whether the first and second contents in the sample pair are related. If the first and second contents are related, the first content can be used as input to retrieve the second content, or it can be the result of retrieving the second content. If the first and second contents are not related, the probability of retrieving the second content through the first content is low, or the probability of retrieving the first content through the second content is low.

[0056] S202: Obtain a first vector for identifying the first content and a second vector for identifying the second content.

[0057] A vector is a mathematical representation of content. For example, the mathematical representation of the first piece of content is the first vector, and the mathematical representation of the second piece of content is the second vector. Vectors can be obtained through models such as machine learning, or by converting content into a mathematical representation as a vector according to set rules.

[0058] It should be noted that the first vector includes N first-channel features, and the second vector includes M second-channel features, where N is an integer greater than 1 and M is an integer greater than 1. That is, the vector includes multiple channel features; for example, the first vector includes multiple first-channel features, and the second vector includes multiple second-channel features. Channel features are used to represent content from different perspectives, so that different channel features represent different information. For example, if the first content is a picture of a little boy kicking a ball, then the multiple first-channel features included in the first vector could be first-channel features representing the little boy, first-channel features representing the ball, first-channel features representing the weather, etc.

[0059] Since different channel features represent different information, if positional multiplication is used, the multiple channel features included in different vectors can only interact once, or rather, the interaction occurs between the overall vectors, without considering the interaction between channel features within the vectors. If positional multiplication is used to obtain the similarity between the first and second vectors, this calculation method is sparse, with fewer interactions between the multiple channel features of different vectors, thus failing to effectively explore the relationship between the first and second vectors.

[0060] Furthermore, among multiple channel features, there exists a typical channel feature, which is the most representative of the vector and thus has saliency. The typical channel features of different vectors may be located in different positions within the vector. For example, the typical channel features in the first vector and the typical channel features in the second vector may be located in different positions, indicating a misalignment problem. Through positional multiplication, the typical channel features of the first vector and the typical channel features of the second vector may not interact, resulting in dissimilar results when the first and second vectors, which should be similar, are multiplied positionally.

[0061] Based on this, in order to better explore the relationship between the first vector and the second vector, this embodiment of the application trains an initial retrieval model to increase the number of interactions between the multiple channel features included in different vectors, so as to better communicate the different information represented by the different channel features, thereby strengthening the information exchange between the first vector and the second vector and better exploring the relationship between the first vector and the second vector. The training process of the initial retrieval model is described in detail below.

[0062] S203: Input the first vector and the second vector into the initial retrieval model. Based on the correlation between each first channel feature and at least two second channel features, the initial retrieval model obtains L undetermined similarities between the first vector and the second vector.

[0063] Among them, compared to the retrieval model, the parameters of the initial retrieval model have not yet been trained.

[0064] The first and second vectors are input into the initial retrieval model. Instead of obtaining the similarity between the first and second vectors through positional multiplication, the initial retrieval model obtains L undetermined similarities between the first and second vectors based on the correlation between each first channel feature and at least two second channel features, where L is an integer greater than 1. For example... Figure 1 As shown, each first channel feature interacts with two second channel features to obtain two undetermined similarities.

[0065] Compared to the positional multiplication method, the interaction between multiple channel features included in different vectors is increased through at least two interactions. This increases the number of interactions between multiple channel features included in different vectors, thus enabling attention to the interactions within different vectors and increasing the density of relationships between channel features. As a result, the relationship between the first and second vectors can be better explored through continuous internal interactions.

[0066] Furthermore, by increasing the number of interactions between multiple channel features, the probability of typical channel feature interactions can be increased, thereby improving the accuracy of the similarity between the first vector and the second vector.

[0067] S204: Take the largest of the L undetermined similarities as the similarity between the first vector and the second vector.

[0068] By increasing the number of interactions between multiple channel features included in different vectors, various relationships between the first and second vectors can be obtained. Successful interactions between more important channel features can improve the similarity between the first and second vectors. Thus, the best performance of the first and second vectors, i.e., the largest undetermined similarity, is taken as the final similarity result, thereby improving the accuracy of the similarity.

[0069] S205: Based on the similarity between the first vector and the second vector, and the sample labels, train the parameters of the initial retrieval model to obtain the retrieval model.

[0070] It should be noted that if the sample labels indicate a relationship between the first and second content items, the greater the similarity between the first and second vectors; conversely, if the sample labels indicate no relationship between the first and second content items, the smaller the similarity between the first and second vectors. Based on this, the parameters of the initial retrieval model can be adjusted to obtain a trained retrieval model. This retrieval model is used to calculate the similarity between different content items. For example, if a user inputs keyword C and the database contains content D to be displayed, the retrieval model can obtain the similarity between keyword C and content D, and thus determine whether to display content D as a search result for keyword C based on the similarity. For example, if the similarity between keyword C and content D is 98%, then content D will be displayed to the user as a search result for keyword C.

[0071] As can be seen from the above technical solution, a sample pair consisting of first content and second content is obtained, and this sample pair has sample labels that identify whether the first content and the second content are related. A first vector for identifying the first content and a second vector for identifying the second content are obtained. The first vector includes multiple first channel features, and the second vector includes multiple second channel features. Since the information represented by each channel feature is different, in order to allow the first vector and the second vector to interact better, the first vector and the second vector are input into an initial retrieval model. The initial retrieval model obtains multiple undetermined similarities between the first vector and the second vector based on the correlation between each first channel feature and at least two second channel features, increasing the number of interactions between the multiple channel features included in different vectors and strengthening the information exchange between the first vector and the second vector. Using the largest undetermined similarity among multiple undetermined similarities as the similarity between the first vector and the second vector is more accurate than the similarity obtained through only one vector interaction. Therefore, by training the parameters of the initial model with more accurate similarities and sample labels, the accuracy of the obtained retrieval model is higher. Therefore, by allowing multiple channel features between different vectors to interact, the number of interactions between channel features between vectors is increased, thereby strengthening the information exchange between different vectors. This makes the similarity between different vectors more accurate, and the retrieval model trained based on the more accurate similarity is more accurate and has better generalization performance.

[0072] As one possible implementation, this application provides a specific implementation method in S203, in which the initial retrieval model obtains L undetermined similarities between the first vector and the second vector based on the correlation between each first channel feature and at least two second channel features, as detailed in A1-A3:

[0073] A1: Using the initial retrieval model, roll the M second channel features in the second vector K times according to the preset step size to obtain K second rolling vectors, and use the K second rolling vectors and the second vector as the second vector set.

[0074] It should be noted that the first vector and the second vector contain the same number of channel features.

[0075] This application does not specifically limit the size of the preset step size, such as the position of one channel feature. Those skilled in the art can set it according to actual needs. The positions of each first channel feature in the first vector are fixed, and then M second channel features in the second vector are rolled according to the preset step size. After rolling K times, K second rolling vectors are obtained. Each second rolling vector is the same as the second vector, including M second channel features, and K is a positive integer.

[0076] For example, see Figure 3This figure is a schematic diagram illustrating a method for determining the similarity of objects to be determined by a scrolling mechanism, as provided in an embodiment of this application. Figure 3 In the first vector, there are 4 first channel features, and the second vector includes 4 second channel features. The first feature vector is fixed and each first channel feature is stationary. The second feature vector at the first position in the second vector is continuously rolled backward until the positions of all channel features have been traversed, resulting in 3 second rolling vectors.

[0077] As one possible implementation, multiple first channel features can be rolled together according to a preset step size to improve the rolling speed and calculation speed.

[0078] A2: Based on the vectors in the first and second vector sets, determine the correlation of the first channel feature according to the position of the channel feature.

[0079] The second vector set includes K second rolling vectors and a second vector, meaning the vectors in the second vector set are K second rolling vectors and a second vector. The correlation of the nth first channel feature in the first vector is determined based on the nth first channel feature and the nth second channel feature of each vector in the second vector set, where n is an integer less than N.

[0080] Continue with Figure 3 For example, the correlation of the second first channel feature in the first vector is determined based on the second second channel feature of each vector in the set of the second vector. That is, the position between the channel features included in different vectors is determined, and the position of the same channel feature is determined as the correlation. Then, L undetermined similarities between the first vector and the second vector are obtained based on the positional multiplication method.

[0081] A3: Based on the correlation of the first channel features, obtain L undetermined similarities between the first vector and the second vector.

[0082] This application does not specifically limit the method of calculating similarity in its embodiments; those skilled in the art can set it according to actual needs. Continuing with... Figure 3 For example, the vectors in the first and second vector sets are input into the Weighted Token-wise (WTI) method to calculate the similarity between the vectors, resulting in 4 undetermined similarities.

[0083] It should be noted that, in Figure 3 In this process, the largest of the four undetermined similarities is taken as the similarity between the first vector and the second vector.

[0084] Therefore, by fixing the N first-channel features of the first vector and rolling the M second-channel features of the second vector according to a preset step size, a second vector set is obtained. The vectors in the first and second vector sets are then multiplied positionally to obtain multiple undetermined similarities. The largest undetermined similarity among these is then used as the similarity between the first and second vectors. Compared to the positional multiplication method in related technologies, obtaining multiple rolling vectors first and then performing positional multiplication increases the interaction between the N first-channel features of the first vector and the M second-channel features of the second vector, even enabling full interaction between them. This further increases the number of interactions between the multiple channel features of different vectors, allowing attention to the interactions within different vectors and increasing the density of relationships between channel features. Thus, through continuous interaction within different vectors, the relationship between the first and second vectors can be better explored.

[0085] Furthermore, not only can the position of the channel features of the first vector be fixed while the second vector is scrolled, but also the position of the channel features of the second vector can be fixed while the first vector is scrolled. See B1-B5 for details:

[0086] B1: Using the initial retrieval model, roll the M second channel features in the second vector K times according to the preset step size to obtain K second rolling vectors, and use the K second rolling vectors and the second vector as the second vector set.

[0087] B2: Based on the vectors in the first and second vector sets, determine the correlation of the first channel feature according to the position of the channel feature.

[0088] The relevant details are as described in A1 and A2 above, and will not be repeated here.

[0089] B3: Using the initial retrieval model, roll the N first channel features in the first vector K times according to the preset step size to obtain K first rolling vectors, and use the K first rolling vectors and the first vector as the first vector set.

[0090] Referring to A1 above, through the initial retrieval model, the positions of each second channel feature in the second vector are fixed, and N second channel features in the first vector are rolled according to a preset step size. After rolling K times, K first rolling vectors are obtained. Each first rolling vector is the same as the first vector, including N second channel features, where K is a positive integer.

[0091] B4: Based on the vectors in the first vector set and the second vector, determine the correlation for the second channel feature according to the position of the channel feature.

[0092] Referring to A2 above, the correlation of the m-th second channel feature in the second vector is determined based on the m-th second channel feature and the m-th first channel feature of each vector in the set of the first vector, where m is an integer less than M.

[0093] B5: Based on the correlation between the first channel features and the second channel features, obtain L undetermined similarities between the first vector and the second vector.

[0094] The following explanation uses formulas (1) to (5) to illustrate the specifics.

[0095] WTI(V i ,T i )=(t2v_logits+v2t_logits) / 2 (1)

[0096]

[0097]

[0098] Among them, f tw,θ and f vw,θ It is a classic MLP structure with SoftMax, and L is a parameter that needs to be tuned in the initial retrieval model. T L is the number of the first subvectors included in the first vector. V The second vector contains the number of second sub-vectors, where i represents the sample index, such as T. i V is the i-th first vector. i Let p be the index of the first sub-vector in the first vector, and q be the index of the second sub-vector in the second vector.

[0099] Sim k =WTI(T,r) k (V)),k∈(0,d],N * (4)

[0100] Sim = Max(Sim) k ),k∈(0,d],N * (5)

[0101] Where r(·) represents the rolling function, which rolls the matrix elements along the channel feature axis, and k represents the k-th roll. Sim k Let be the k-th undetermined similarity, Max(·) represents taking the maximum value bit by bit, Sim is the similarity between the first vector and the second vector, and d is the number of feature channels.

[0102] In retrieval tasks, the input and output content are similar (or the tasks are complementary). If the first and second content are related, then they are similar. Therefore, we can not only fix the position of the channel features of the first vector while scrolling the second vector, but also fix the position of the channel features of the second vector while scrolling the first vector. By calculating the similarity between two complementary content sets, we can improve the accuracy of the similarity calculation. This allows for a more accurate retrieval model trained based on a more precise similarity score.

[0103] As one possible implementation, this application provides a specific implementation method in S203, in which the initial retrieval model obtains L undetermined similarities between the first vector and the second vector based on the correlation between each first channel feature and at least two second channel features, as detailed in C1-C3:

[0104] In this embodiment, the first vector includes I first sub-vectors, each of which includes N first channel features; the second vector includes J second sub-vectors, each of which includes M second channel features. For example, if the first vector is a text vector, then the text vector includes I first sub-vectors corresponding to each word segment; if the second vector is a video vector, then the video vector includes J second sub-vectors corresponding to each video frame.

[0105] C1: For the first and second sub-vectors of the target, the initial retrieval model obtains the undetermined similarity between the first and second sub-vectors of the target based on the correlation between the N first channel features included in the first sub-vector and the M second channel features included in the second sub-vector.

[0106] Here, the first target sub-vector is one of I first sub-vectors, and the second target sub-vector is one of J second sub-vectors. The final undetermined similarity is obtained by multiplying all first-channel features included in the first target sub-vector with all second-channel features included in the second target sub-vector.

[0107] For example, see Figure 4 This figure illustrates a method for determining undetermined similarity through full-channel multiplication, as provided in an embodiment of this application. The first vector comprises three first sub-vectors, each containing four first-channel features. The second vector comprises three second sub-vectors, each containing four second-channel features. In the large nine-square grid, each cell represents the undetermined similarity between one first sub-vector and one second sub-vector.

[0108] Taking the top-left cell of the 3x3 grid as an example, for the first and second target sub-vectors, the initial retrieval model obtains the undetermined similarity between the first and second target sub-vectors based on the correlation between the four first-channel features included in the first sub-vector and the four second-channel features included in the second sub-vector. For example, the area containing the triangle represents the product of the second first-channel feature included in the first target sub-vector and the first second-channel feature included in the second target sub-vector, yielding the undetermined similarity between the second first-channel feature and the first second-channel feature. This results in the undetermined similarity among 16 channel features, and finally, the undetermined similarity between the first and second target sub-vectors is obtained based on the average of these 16 undetermined similarities.

[0109] C2: Take I first sub-vectors as target first sub-vectors and J second sub-vectors as target second sub-vectors to obtain I*J undetermined similarities between the first and second sub-vectors, where L is the product of I and J.

[0110] See also Figure 4 By taking each first sub-vector as the target first sub-vector and each second sub-vector as the target second sub-vector, the undetermined similarity among the nine sub-vectors was obtained.

[0111] C3: Based on the I*J undetermined similarities between the first sub-vector and the second sub-vector, determine the L undetermined similarities between the first vector and the second vector.

[0112] Therefore, by interacting with all channel features included in a vector, the number of interactions between multiple channel features included in different vectors is greatly increased. This allows attention to the interactions within different vectors, increases the density of relationships between channel features, and ensures that typical channel features between vectors can interact. Thus, by exploring all interactions within different vectors, the relationship between the first and second vectors can be better discovered. Compared to the aforementioned method of determining the undetermined similarity through a rolling approach, this application has lower computational complexity.

[0113] Furthermore, this application provides a specific implementation method for C3, which determines L undetermined similarities between the first and second vectors based on I*J undetermined similarities between the first and second sub-vectors, as shown in D1-D2:

[0114] D1: For the i-th first sub-vector among I first sub-vectors, take the largest similarity among the J undetermined similarities between the i-th first sub-vector and the second sub-vector as the similarity between the i-th first sub-vector and the second sub-vector.

[0115] Where i is a positive integer less than I. Continuing with... Figure 4For example, arrange the I*J undetermined similarities in matrix form (I rows and J columns). The circled area represents the undetermined similarity between the second second sub-vector and the first first sub-vector. That is, looking at it row-wise, the first row is the similarity between the first first sub-vector and each second sub-vector; looking at it column-wise, the first column is the similarity between the first second sub-vector and each first sub-vector. Then D1 takes the maximum value in each row as the similarity between the i-th first sub-vector and the second sub-vector, obtaining the similarity to the right of the I*J undetermined similarities.

[0116] D2: Determine the undetermined similarity between the first vector and the second vector based on the similarity between the I first sub-vectors and the second sub-vectors respectively.

[0117] Continue with Figure 4 For example, arrange the I*J undetermined similarities in matrix form, and take the maximum similarity in each row as the similarity between the i-th first sub-vector and the second sub-vector. Then, based on the similarities between the I first sub-vectors and the second sub-vectors, determine the undetermined similarity between the first and second vectors. Alternatively, take the maximum similarity in each column as the similarity between the j-th second sub-vector and the first sub-vector. Then, based on the similarities between the J second sub-vectors and the first sub-vector, determine the undetermined similarity between the first and second vectors.

[0118] Furthermore, not only can the undetermined similarity between the first and second vectors be determined based on the similarity between each of the I first sub-vectors and the second sub-vector, but the similarity between the first and second vectors can also be determined based on the similarity between the J second sub-vectors and the first sub-vector. See E1-E3 for details:

[0119] E1: For the i-th first sub-vector among I first sub-vectors, take the largest similarity among the J undetermined similarities between the i-th first sub-vector and the second sub-vector as the similarity between the i-th first sub-vector and the second sub-vector.

[0120] For relevant details, please refer to D1 above, which will not be repeated here.

[0121] E2: For the j-th second sub-vector among J second sub-vectors, take the largest similarity among the I undetermined similarities between the j-th second sub-vector and the first sub-vector as the similarity between the j-th second sub-vector and the first sub-vector.

[0122] Where j is a positive integer less than J. Continuing with... Figure 4Taking [Example] as an example, E1 takes the maximum value in each row as the similarity between the i-th first sub-vector and the second sub-vector, obtaining the similarities (columns) on the right side of I*J undetermined similarities; E2 takes the maximum value in each column as the similarity between the j-th second sub-vector and the first sub-vector, obtaining the similarities (rows) above I*J undetermined similarities.

[0123] E3: Determine the similarity between the first vector and the second vector according to the similarities between the I first sub-vectors and the second sub-vectors respectively, and the similarities between the J second sub-vectors and the first sub-vectors respectively.

[0124] For example, the similarity between the first vector and the second vector can be directly determined according to the similarities between the I first sub-vectors and the second sub-vectors respectively, and the similarities between the J second sub-vectors and the first sub-vectors respectively.

[0125] For another example, first obtain the first weights of the I first sub-vectors in the first vector respectively, and the second weights of the J second sub-vectors in the second vector respectively, and then determine the similarity between the first vector and the second vector according to the similarities between the I first sub-vectors and the second sub-vectors respectively, the first weights, the similarities between the J second sub-vectors and the first sub-vectors respectively, and the second weights.

[0126] Continue to refer to Figure 4

[0123] E3: Determine the similarity between the first vector and the second vector according to the similarities between the I first sub-vectors and the second sub-vectors respectively, and the similarities between the J second sub-vectors and the first sub-vectors respectively.

[0124] For example, the similarity between the first vector and the second vector can be directly determined according to the similarities between the I first sub-vectors and the second sub-vectors respectively, and the similarities between the J second sub-vectors and the first sub-vectors respectively.

[0125] For another example, first obtain the first weights of the I first sub-vectors in the first vector respectively, and the second weights of the J second sub-vectors in the second vector respectively, and then determine the similarity between the first vector and the second vector according to the similarities between the I first sub-vectors and the second sub-vectors respectively, the first weights, the similarities between the J second sub-vectors and the first sub-vectors respectively, and the second weights.

[0126] Continue to refer to Figure 4 As shown in [Figure], the 3 similarities between the three first sub-vectors and the second sub-vector are multiplied by the corresponding first weights respectively, and the 3 similarities between the three second sub-vectors and the first sub-vector are multiplied by the corresponding second weights respectively. The sum of the two multiplication results is divided by 2 to obtain the similarity between the first vector and the second vector.

[0127] Thus, multiplying the similarity by the corresponding weight is to remove the repeatedly occurring and meaningless information. For example, in the text content, the word "of" appears frequently, but it contributes less to the meaning expressed by the text content. Therefore, the weight corresponding to "of" can be set smaller to avoid the influence of repeatedly occurring and meaningless information on the similarity and improve the accuracy of the similarity.

[0128] This application embodiment does not specifically limit the acquisition methods of the first weight and the second weight, and those skilled in the art can set them according to actual needs or obtain them by training other models.

[0129] Next, E1 - E3 will be further described through formulas (6) - (9).

[0130]

[0131]

[0132]

[0133] Similarity=(t2v_logits+v2t_logits) / 2 (9)

[0134] Among them, f tw,θ and f vw,θ It is a classic MLP structure with SoftMax, and L is a parameter that needs to be tuned in the initial retrieval model. T L is the number of the first subvectors included in the first vector. V The second vector contains the number of second sub-vectors, where i represents the sample index, such as T. i V is the i-th first vector. i Let p be the index of the first sub-vector in the first vector, and q be the index of the second sub-vector in the second vector. Mean represents multiplying all channels, while Mean represents averaging.

[0135] In retrieval tasks, the input and output content are similar (or the tasks are complementary). If the first and second content are related, then they are similar. Therefore, the similarity between the first and second vectors can be determined based on the similarity between I first sub-vectors and the second sub-vectors, and the similarity between J second sub-vectors and the first sub-vectors. Calculating the similarity between two complementary tasks improves the accuracy of the similarity score, allowing for a more accurate retrieval model trained on this more precise similarity data.

[0136] As one possible implementation, this application provides an S205, which is to obtain a specific implementation of the retrieval model based on the similarity between the first vector and the second vector, and the parameters of the initial retrieval model trained by the sample labels, as detailed in F1-F4:

[0137] F1: Obtain the B pairs of samples required for training the p-th initial retrieval model, and the similarity between the B first vectors and the second vectors obtained by the initial retrieval model for the B pairs of samples.

[0138] Where p is a positive integer and B is an integer greater than 1. During the training of the initial retrieval model, a batch of sample pairs is used to train the initial retrieval model each time. For example, when training the initial retrieval model for the pth time, B sample pairs are used as a batch of sample pairs for training.

[0139] For example, obtain the first vector of the first content and the second vector of the second content included in the B pair of samples. Input the first vector and the second vector belonging to a pair of samples into the initial retrieval model respectively. The similarity between the first vector and the second vector of the B pair is obtained through the initial retrieval model, that is, the similarity between the B pair of samples.

[0140] F2: Determine the similarity between the first sample pair and other sample pairs.

[0141] Among them, the first sample pair is a pair of positive sample pairs, and the positive sample pair is a pair of sample labels in the B-pair sample pairs that represent the correlation between the first content and the second content. The other sample pairs are the sample pairs in the B-pair sample pairs excluding the first sample pair.

[0142] For example, select one pair of related sample pairs as the first sample pair, and select the other sample pairs in the B sample pairs except for the first sample pair as the other sample pairs.

[0143] This application does not specifically limit the similarity of the first sample to other sample pairs. The following uses the data type of the first content as text and the data type of the second content as video as an example to explain the calculation method of the similarity between the first sample pair and other sample pairs in two ways.

[0144] Method 1: Obtain the second content of the first sample pair and the second content of the second sample pair, where the second sample pair is one of the other sample pairs. Extract any frame of the second content of the first sample pair and any frame of the second content of the second sample pair. Calculate the similarity between the two frames and use the obtained similarity as the similarity between the first sample pair and the second sample pair. This will determine the similarity between the first sample pair and other sample pairs.

[0145] However, Method 1 has the following problems: (1) Randomly selecting two video frames does not fully establish the relationship between the two videos; (2) The video vectors corresponding to the video content contain a lot of noise and have unclear semantics, making it impossible to mine potential similar sample pairs. Based on this, the embodiments of this application also propose Method 2, as follows:

[0146] Method 2: Obtain the first text vector and the second text vector; determine the similarity between the first text vector and the second text vector based on the first text vector and the second text vector, and determine the similarity between the first text vector and the second text vector as the sample similarity between the first sample pair and the second sample pair; take the sample pairs in other samples as the second sample pairs respectively, and determine the sample similarity between the first sample pair and other sample pairs.

[0147] In this method, the first text vector is the first vector of the first content in the first sample pair, the second text vector is the first vector of the first content in the second sample pair, and the second sample pair is one of the other sample pairs. Compared to method one, method two no longer determines the similarity between text-video pairs (sample pairs) through video content, but instead uses text content to determine the similarity. The text vectors corresponding to the text content are virtually free of noise and have clearer semantics. Moreover, compared to video vectors, text vectors contain less content, allowing for more computational use while carrying more semantic information, resulting in more accurate text-video pair similarity and thus more accurate subsequent retrieval models.

[0148] As one possible approach, the similarity between sample pairs can also be determined by using a fused vector of video and text vectors.

[0149] F3: If the similarity between the first sample pair and other sample pairs meets the similarity threshold, then the sample pairs that meet the similarity threshold are added to the positive sample set of the first sample pair; if the similarity between the first sample pair and other sample pairs does not meet the similarity threshold, then the sample pairs that do not meet the similarity threshold are added to the negative sample set of the first sample pair.

[0150] If the similarity between the first sample pair and other sample pairs meets the similarity threshold, it means that the first sample pair is relatively similar to the other sample pairs. Therefore, sample pairs that meet the similarity threshold can be added to the positive sample set of the first sample pair. It should be noted that the positive sample set of the first sample pair includes the first sample pair itself. If the similarity between the first sample pair and other sample pairs does not meet the similarity threshold, it means that the first sample pair is not very similar to the other sample pairs. Therefore, sample pairs that do not meet the similarity threshold can be added to the negative sample set of the first sample pair.

[0151] This application does not specifically limit the size of the similarity threshold; those skilled in the art can set it according to actual needs. For example, the similarity threshold can be as shown in formula (10):

[0152]

[0153] Where, n i Let n be the i-th first sub-vector in the first vector of the first sample pair. j It is the j-th first sub-vector in the first vector of other sample pairs. Let n be the number of permutations and combinations. i There are a total of first subvectors The combination method. Let be the cosine similarity between the k-th first sub-vector of the i-th first vector in the first sample pair and the l-th first sub-vector of the j-th first vector in another sample pair.

[0154] F4: Take each of the B sample pairs as the first sample pair, and train the parameters of the initial retrieval model based on the training direction of maximizing the similarity of sample pairs in the positive sample set of the first sample pair and minimizing the similarity of sample pairs in the negative sample set of the first sample pair, thus obtaining the retrieval model.

[0155] In related technologies, during the p-th training iteration, a first content corresponds to only one positive second content. For example, the first content and the second content in the first sample pair form a positive sample pair, and then all the second content in other sample pairs form negative sample pairs with the first content in the first sample pair. The retrieval model is then trained. For example, formula (11) can be used as the loss function for training. Formula (11) is as follows:

[0156]

[0157] Where exp(·) represents the exponential function, WTI is the similarity calculation method, log(·) is the logarithmic calculation method, B represents the total number of samples, and i represents the index of the sample in the B sample pairs.

[0158] However, this approach has a drawback: all the second content in other sample pairs can form positive sample pairs because they are similar to the first content, instead of being mechanically treated as negative sample pairs. If the first and second content, which are originally quite similar, are used as negative sample pairs for training, it will impair the performance of the retrieval model.

[0159] Based on this, the embodiments of this application determine the positive sample set and the negative sample set by calculating the similarity between sample pairs, which can remove noise in the sample labels, thereby further improving the accuracy of similarity calculation. Then, based on the training direction of maximizing the similarity of sample pairs in the positive sample set of the first sample pair and minimizing the similarity of sample pairs in the negative sample set of the first sample pair, the parameters of the initial retrieval model are trained to obtain a more accurate retrieval model.

[0160] For example, in embodiments of this application, the loss function shown in formula (12) can be used for training.

[0161]

[0162] Where exp(·) represents the exponential function, WTI is the similarity calculation method, log(·) is the logarithmic calculation method, B represents the total number of samples, and i represents the index of the sample in the B sample pairs. Represents the set of positive samples. This represents the set of negative samples.

[0163] As one possible implementation, embodiments of this application also provide a way to use the retrieval model, as detailed in G1-G3:

[0164] G1: Retrieve search results and content to be displayed.

[0165] In either case, the searched content is the first content, and the content to be displayed is the second content; or, the searched content is the second content, and the content to be displayed is the first content.

[0166] G2: Input the search content and the content to be displayed into the search model to obtain the similarity between the search content and the content to be displayed.

[0167] G3: If the similarity between the searched content and the content to be displayed meets the threshold condition, the content to be displayed will be used as the search result of the searched content and displayed.

[0168] The embodiments of this application do not specifically limit the threshold conditions. Those skilled in the art can set them according to actual needs, such as the similarity between the searched content and the content to be displayed exceeding 80%, or the similarity between the searched content and the content to be displayed ranking in the top 10.

[0169] Therefore, the trained retrieval model can perform content retrieval and display the corresponding search results to the user.

[0170] As one possible implementation, the first content and the second content are of different data types, or in other words, the first content and the second content belong to different modalities of data. For example, the first content is text content, and the second content is non-text content, such as video content, audio content, image content, etc. Thus, the retrieval model trained accordingly can be used to retrieve video content, audio content, image content, etc. through text content retrieval.

[0171] Compared to searching between different content of the same data type, the channel features of different content of different data types are more likely to be misaligned. The retrieval model obtained by using the training method of the retrieval model provided in the embodiments of this application can pay attention to the interaction between the channel features included in different content, so as to obtain more accurate similarity and thus recommend more accurate retrieval results to users, thereby improving the user experience.

[0172] To facilitate a further understanding of the technical solutions provided in the embodiments of this application, the following example uses a server as the execution subject of the training method for the retrieval model provided in the embodiments of this application, and the first content is video content and the second content is text content, to provide an overall exemplary description of the training method for the retrieval model.

[0173] The training process of the retrieval model will be explained below.

[0174] See Figure 5 This figure is a schematic diagram of a scenario illustrating the training and application method of a retrieval model provided in an embodiment of this application. Text content and video content are acquired. The video content is input into a video encoder to obtain video vectors, and the text content is input into a text encoder to obtain text vectors. The video encoder can be a video encoder based on Contrastive Language–Image Pre-training (CLIP), and the text encoder can be a CLIP-based text encoder. The text vectors and video vectors are input into an initial retrieval model to obtain the similarity between the text vectors and video vectors. Then, through the aforementioned F1-F4 implementation of sample label denoising, the positive and negative sample sets of the text vectors are determined, resulting in a similarity matrix of multiple video-text pairs. The top left corner of the similarity matrix represents the similarity between the first text content and the first video content, and the first row of the similarity matrix represents the similarity between the first video content and each text content.

[0175] During the training of the initial retrieval model, the aforementioned formula (12) is set to 0, that is, the loss function is minimized as much as possible, thereby achieving parameter adjustment (parameter tuning) of the initial retrieval model. As mentioned above, the similarity on the diagonal of the similarity matrix is ​​a positive sample pair. That is, the initial retrieval model should be trained based on maximizing the similarity on the diagonal of the similarity matrix, maximizing the similarity of sample pairs similar to the sample pairs on the diagonal, and minimizing the similarity of other sample pairs. In other words, the training direction of the initial retrieval model is the similarity of the sample pairs on the diagonal and the similarity of sample pairs similar to the sample pairs on the diagonal. It is understandable that during the parameter adjustment of the initial retrieval model, the text encoder and video encoder can also be adjusted based on the loss function so that the text encoder and video encoder can learn the text content and video content more deeply. Until the training is completed and the retrieval model is obtained.

[0176] It should be noted that the similarity matrix obtained from the retrieval model can be used to sort the results and obtain the final prediction results. These results can then be used to evaluate the model performance using R@1, R@5, R@10, Median Score, and Mean Score. For example, the similarity matrix viewed by rows sorts the similarity between each text content and each video content (which can be represented as T2V), while the similarity matrix viewed by columns sorts the similarity between each video content and each text content (which can be represented as V2T).

[0177] Here, R@1 represents the first similarity score on the diagonal of its row; R@5 represents the top five similarities on the diagonal of its row; and R@10 represents the top ten similarities on the diagonal of its row. The Median Score and Mean Score are the average scores. The performance of the obtained retrieval model is evaluated using one or more of these five performance metrics.

[0178] The application process of the retrieval model is explained below.

[0179] See also Figure 5 The system obtains the search content input by the user and the content to be displayed in the database. The search content is input into the corresponding encoder to obtain the corresponding vector, and the content to be displayed is input into the corresponding encoder to obtain the corresponding vector. The two vectors are then input into the retrieval model. The retrieval model can obtain the similarity between the search content and the content to be displayed, so as to determine whether the content to be displayed should be used as the retrieval result based on the similarity.

[0180] In addition to the training method for the retrieval model described above, this application also provides a corresponding training device for the retrieval model, so that the training method for the retrieval model can be applied and implemented in practice.

[0181] See Figure 6 This figure is a schematic diagram of the structure of a training device for a retrieval model provided in an embodiment of this application. Figure 6 As shown, the training device 600 of the retrieval model includes: a first acquisition unit 601, a second acquisition unit 602, a first determination unit 603, a second determination unit 604, and a training unit 605;

[0182] The first acquisition unit 601 is used to acquire sample pairs with sample tags, the sample pairs including first content and second content, and the sample tags are used to identify whether the first content and the second content are related;

[0183] The second acquisition unit 602 is used to acquire a first vector for identifying the first content and a second vector for identifying the second content; the first vector includes N first channel features and the second vector includes M second channel features, where N is an integer greater than 1 and M is an integer greater than 1.

[0184] The first determining unit 603 is used to input the first vector and the second vector into an initial retrieval model, and obtain L undetermined similarities between the first vector and the second vector based on the correlation between each first channel feature and at least two second channel features; L is an integer greater than 1.

[0185] The second determining unit 604 is used to take the largest undetermined similarity among the L undetermined similarities as the similarity between the first vector and the second vector;

[0186] The training unit 605 is used to train the parameters of the initial retrieval model based on the similarity between the first vector and the second vector, and the sample labels, to obtain the retrieval model.

[0187] As one possible implementation, if N equals M, the first determining unit 603 is specifically used for:

[0188] Using the initial retrieval model, the M second channel features in the second vector are rolled K times according to a preset step size to obtain K second rolling vectors. The K second rolling vectors and the second vector are used as a second vector set. The second rolling vector includes M second channel features, and K is a positive integer.

[0189] Based on the first vector and the vectors in the second vector set, the correlation of the first channel feature is determined according to the position of the channel feature. The correlation of the nth first channel feature in the first vector is determined based on the nth first channel feature and the nth second channel feature of each vector in the second vector set, where n is an integer less than N.

[0190] Based on the correlation of the first channel features, L undetermined similarities are obtained between the first vector and the second vector.

[0191] As one possible implementation, after obtaining the K second rolling vectors, the first determining unit 603 is further configured to:

[0192] Using the initial retrieval model, the N first channel features in the first vector are rolled K times according to the preset step size to obtain K first rolling vectors. The K first rolling vectors and the first vector are used as a first vector set, and the first rolling vector includes N first channel features.

[0193] Based on the vectors in the first vector set and the second vector, the correlation for the second channel feature is determined according to the position of the channel feature. The correlation of the m-th second channel feature in the second vector is determined based on the m-th second channel feature and the m-th first channel feature of each vector in the first vector set, where m is an integer less than M.

[0194] Based on the correlation between the first channel features and the second channel features, L undetermined similarities are obtained between the first vector and the second vector.

[0195] As one possible implementation, the first vector includes I first sub-vectors, each of which includes N first channel features; the second vector includes J second sub-vectors, each of which includes M second channel features; and the first determining unit 603 is specifically used for:

[0196] For the first target sub-vector and the second target sub-vector, the initial retrieval model obtains the undetermined similarity between the first target sub-vector and the second target sub-vector based on the correlation between the N first channel features included in the first target sub-vector and the M second channel features included in the second target sub-vector; the first target sub-vector is one of I first sub-vectors, and the second target sub-vector is one of J second sub-vectors;

[0197] I first sub-vectors are used as the target first sub-vectors, and J second sub-vectors are used as the target second sub-vectors, to obtain I*J undetermined similarities between the first sub-vectors and the second sub-vectors;

[0198] Based on the I*J undetermined similarities between the first sub-vector and the second sub-vector, determine L undetermined similarities between the first vector and the second vector.

[0199] As one possible implementation, the first determining unit 603 is specifically used for:

[0200] For the i-th first sub-vector among I first sub-vectors, the largest similarity among the J undetermined similarities between the i-th first sub-vector and the second sub-vector is taken as the similarity between the i-th first sub-vector and the second sub-vector; i is a positive integer less than I;

[0201] Based on the similarity between the first sub-vector and the second sub-vector, the undetermined similarity between the first vector and the second vector is determined.

[0202] As one possible implementation, the first determining unit 603 is specifically used for:

[0203] For the j-th second sub-vector among J second sub-vectors, the largest similarity among the I undetermined similarities between the j-th second sub-vector and the first sub-vector is taken as the similarity between the j-th second sub-vector and the first sub-vector; j is a positive integer less than J;

[0204] The step of determining the undetermined similarity between the first vector and the second vector based on the similarity between each of the first sub-vectors and the second sub-vector includes:

[0205] The similarity between the first vector and the second vector is determined based on the similarity between I first sub-vectors and the second sub-vectors respectively, and the similarity between J second sub-vectors and the first sub-vectors.

[0206] As one possible implementation, the training device 600 for the retrieval model further includes a third acquisition unit, used for:

[0207] Obtain the first weights of I first sub-vectors in the first vector, and the second weights of J second sub-vectors in the second vector;

[0208] The first determining unit 603 is specifically used for:

[0209] The similarity between the first vector and the second vector is determined based on the similarity between the I first sub-vectors and the second sub-vectors, the first weight, the similarity between the J second sub-vectors and the first sub-vectors, and the second weight.

[0210] As one possible implementation, the training unit 605 is specifically used for:

[0211] Obtain B pairs of sample pairs required for training the initial retrieval model for the pth time, and B similarities between the first vector and the second vector obtained by the initial retrieval model for the sample pairs; B is an integer greater than 1;

[0212] The step of training the parameters of the initial retrieval model based on the similarity between the first vector and the second vector, and the sample labels, to obtain the retrieval model includes:

[0213] Determine the similarity between the first sample pair and other sample pairs; the first sample pair is a pair of positive sample pairs, the positive sample pair is a pair of sample pairs in the sample pair in which the sample label represents the correlation between the first content and the second content, and the other sample pairs are the sample pairs in the sample pairs in the sample pair excluding the first sample pair;

[0214] If the similarity between the first sample pair and the other sample pairs meets the similarity threshold, then the sample pairs that meet the similarity threshold are added to the positive sample set of the first sample pair; if the similarity between the first sample pair and the other sample pairs does not meet the similarity threshold, then the sample pairs that do not meet the similarity threshold are added to the negative sample set of the first sample pair.

[0215] Using B pairs of samples as the first sample pairs, the parameters of the initial retrieval model are trained based on the training direction of maximizing the similarity of sample pairs in the positive sample set of the first sample pairs and minimizing the similarity of sample pairs in the negative sample set of the first sample pairs, to obtain the retrieval model. The positive sample set of the first sample pairs includes the first sample pairs.

[0216] As one possible implementation, the training unit 605 is specifically used for:

[0217] Obtain a first text vector and a second text vector, wherein the first text vector is the first vector of the first content in the first sample pair, the second text vector is the first vector of the first content in the second sample pair, and the second sample pair is one of the other sample pairs;

[0218] Based on the first text vector and the second text vector, the similarity between the first text vector and the second text vector is determined, and the similarity between the first text vector and the second text vector is determined as the sample similarity between the first sample pair and the second sample pair;

[0219] The sample pairs in the other samples are used as the second sample pairs to determine the sample similarity between the first sample pair and the other sample pairs.

[0220] As one possible implementation, the training device 600 for the retrieval model further includes an application unit for:

[0221] Obtain the search content and the content to be displayed, wherein the search content is the first content and the content to be displayed is the second content; or, the search content is the second content and the content to be displayed is the first content.

[0222] The search content and the content to be displayed are input into the search model to obtain the similarity between the search content and the content to be displayed;

[0223] If the similarity between the searched content and the content to be displayed meets the threshold condition, the content to be displayed will be used as the search result of the searched content and displayed.

[0224] As one possible implementation, the first content and the second content have different data types.

[0225] As can be seen from the above technical solution, a sample pair consisting of first content and second content is obtained, and this sample pair has sample labels that identify whether the first content and the second content are related. A first vector for identifying the first content and a second vector for identifying the second content are obtained. The first vector includes multiple first channel features, and the second vector includes multiple second channel features. Since the information represented by each channel feature is different, in order to allow the first vector and the second vector to interact better, the first vector and the second vector are input into an initial retrieval model. The initial retrieval model obtains multiple undetermined similarities between the first vector and the second vector based on the correlation between each first channel feature and at least two second channel features, increasing the number of interactions between the multiple channel features included in different vectors and strengthening the information exchange between the first vector and the second vector. Using the largest undetermined similarity among multiple undetermined similarities as the similarity between the first vector and the second vector is more accurate than the similarity obtained through only one vector interaction. Therefore, by training the parameters of the initial model with more accurate similarities and sample labels, the accuracy of the obtained retrieval model is higher. Therefore, by allowing multiple channel features between different vectors to interact, the number of interactions between channel features between vectors is increased, thereby strengthening the information exchange between different vectors. This makes the similarity between different vectors more accurate, and the retrieval model trained based on the more accurate similarity is more accurate and has better generalization performance.

[0226] This application also provides a computer device, which is the aforementioned computer device. This computer device can be a server or a terminal device, and the training device for the aforementioned retrieval model can be built into the server or terminal device. The computer device provided in this application will be described below from a hardware implementation perspective. Figure 7 The diagram shown is a structural schematic of the server. Figure 8 The diagram shown is a structural schematic of the terminal device.

[0227] See Figure 7This figure is a schematic diagram of a server structure provided in an embodiment of this application. The server 1400 can vary considerably due to different configurations or performance. It may include one or more processors 1422, such as a central processing unit (CPU), memory 1432, and one or more application programs 1442 or data storage media 1430 (e.g., one or more mass storage devices). The memory 1432 and storage media 1430 can be temporary or persistent storage. The program stored in the storage media 1430 may include one or more modules (not shown in the figure), each module may include a series of instruction operations on the server. Furthermore, the processor 1422 may be configured to communicate with the storage media 1430 and execute the series of instruction operations in the storage media 1430 on the server 1400.

[0228] Server 1400 may also include one or more power supplies 1426, one or more wired or wireless network interfaces 1450, one or more input / output interfaces 1458, and / or one or more operating systems 1441, such as Windows Server. TM Mac OS X TM Unix TM Linux TM FreeBSD TM etc.

[0229] The steps performed by the server in the above embodiments can be based on this Figure 7 The server structure shown.

[0230] CPU 1422 is used to perform the following steps:

[0231] Obtain sample pairs with sample tags, wherein the sample pairs include first content and second content, and the sample tags are used to identify whether the first content and the second content are related;

[0232] Obtain a first vector for identifying the first content and a second vector for identifying the second content; the first vector includes N first channel features and the second vector includes M second channel features, where N is an integer greater than 1 and M is an integer greater than 1.

[0233] The first vector and the second vector are input into the initial retrieval model. The initial retrieval model obtains L undetermined similarities between the first vector and the second vector based on the correlation between each first channel feature and at least two second channel features; L is an integer greater than 1.

[0234] The largest of the L undetermined similarities is taken as the similarity between the first vector and the second vector.

[0235] The retrieval model is obtained by training the parameters of the initial retrieval model based on the similarity between the first vector and the second vector, and the sample labels.

[0236] Optionally, the CPU 1422 may also execute method steps of any specific implementation of the training method of the retrieval model in the embodiments of this application.

[0237] See Figure 8 The figure is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Figure 8 This diagram illustrates a partial structure of a smartphone related to the terminal device provided in this embodiment. The smartphone includes components such as a radio frequency (RF) circuit 1510, a memory 1520, an input unit 1530, a display unit 1540, a sensor 1550, an audio circuit 1560, a Wi-Fi module 1570, a processor 1580, and a power supply 1590. Those skilled in the art will understand that... Figure 8 The smartphone structure shown does not constitute a limitation on smartphones and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0238] The following is combined with Figure 8 A detailed introduction to the various components of a smartphone:

[0239] The RF circuit 1510 can be used to receive and transmit signals during information transmission or calls. In particular, it receives downlink information from the base station and processes it with the processor 1580; in addition, it transmits uplink data to the base station.

[0240] The memory 1520 can be used to store software programs and modules, and the processor 1580 runs the software programs and modules stored in the memory 1520 to realize various functions and data processing of the smartphone.

[0241] Input unit 1530 can be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the smartphone. Specifically, input unit 1530 may include touch panel 1531 and other input devices 1532. Touch panel 1531, also known as a touch screen, can collect touch operations on or near the user and drive corresponding connected devices according to a pre-set program. In addition to touch panel 1531, input unit 1530 may also include other input devices 1532. Specifically, other input devices 1532 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0242] The display unit 1540 can be used to display information input by the user or information provided to the user, as well as various menus of the smartphone. The display unit 1540 may include a display panel 1541, which may optionally be configured as a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0243] Smartphones may also include at least one sensor 1550, such as a light sensor, a motion sensor, and other sensors. Other sensors that smartphones may also be equipped with, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be detailed here.

[0244] Audio circuit 1560, speaker 1561, and microphone 1562 provide an audio interface between the user and the smartphone. Audio circuit 1560 converts received audio data into electrical signals and transmits them to speaker 1561, where speaker 1561 converts them into sound signals for output. On the other hand, microphone 1562 converts collected sound signals into electrical signals, which are received by audio circuit 1560, converted into audio data, and then processed by processor 1580 before being transmitted via RF circuit 1510 to, for example, another smartphone, or the audio data can be output to memory 1520 for further processing.

[0245] The processor 1580 is the control center of the smartphone, connecting various parts of the smartphone through various interfaces and lines. It performs various functions and processes data by running or executing software programs and / or modules stored in the memory 1520, and by calling data stored in the memory 1520. Optionally, the processor 1580 may include one or more processing units.

[0246] The smartphone also includes a power supply 1590 (such as a battery) that supplies power to various components. Preferably, the power supply can be logically connected to the processor 1580 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0247] Although not shown, smartphones may also include a camera, Bluetooth module, etc., which will not be described in detail here.

[0248] In this embodiment of the application, the memory 1520 included in the smartphone can store program code and transmit the program code to the processor.

[0249] The processor 1580 included in the smartphone can execute the training method of the retrieval model provided in the above embodiments according to the instructions in the program code.

[0250] This application also provides a computer-readable storage medium for storing a computer program for executing the training method of the retrieval model provided in the above embodiments.

[0251] This application also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the training method for the retrieval model provided in the various optional implementations of the above aspects.

[0252] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium can be at least one of the following media: read-only memory (ROM), RAM, magnetic disk, or optical disk, etc., and other media capable of storing program code.

[0253] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0254] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Based on the implementation methods provided in the above aspects, this application can also be further combined to provide more implementation methods. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for training a retrieval model, characterized in that, The method includes: Obtain sample pairs with sample tags, wherein the sample pairs include first content and second content, and the sample tags are used to identify whether the first content and the second content are related; Obtain a first vector for identifying the first content and a second vector for identifying the second content; the first vector includes N first channel features and the second vector includes M second channel features, where N is an integer greater than 1 and M is an integer greater than 1. The first vector and the second vector are input into the initial retrieval model. The initial retrieval model obtains L undetermined similarities between the first vector and the second vector based on the correlation between each first channel feature and at least two second channel features; L is an integer greater than 1. The largest of the L undetermined similarities is taken as the similarity between the first vector and the second vector. The retrieval model is obtained by training the parameters of the initial retrieval model based on the similarity between the first vector and the second vector, and the sample labels. Wherein, if N equals M, the initial retrieval model obtains L undetermined similarities between the first vector and the second vector based on the correlation between each of the first channel features and at least two of the second channel features, including: Using the initial retrieval model, the M second channel features in the second vector are rolled K times according to a preset step size to obtain K second rolling vectors. The K second rolling vectors and the second vector are used as a second vector set. The second rolling vector includes M second channel features, and K is a positive integer. Based on the first vector and the vectors in the second vector set, the correlation of the first channel feature is determined according to the position of the channel feature. The correlation of the nth first channel feature in the first vector is determined based on the nth first channel feature and the nth second channel feature of each vector in the second vector set, where n is an integer less than N. Based on the correlation of the first channel features, L undetermined similarities are obtained between the first vector and the second vector.

2. The method according to claim 1, characterized in that, After obtaining the K second rolling vectors, the method further includes: Using the initial retrieval model, the N first channel features in the first vector are rolled K times according to the preset step size to obtain K first rolling vectors. The K first rolling vectors and the first vector are used as a first vector set, and the first rolling vector includes N first channel features. Based on the vectors in the first vector set and the second vector, the correlation for the second channel feature is determined according to the position of the channel feature. The correlation of the m-th second channel feature in the second vector is determined based on the m-th second channel feature and the m-th first channel feature of each vector in the first vector set, where m is an integer less than M. The step of obtaining L undetermined similarities between the first vector and the second vector based on the correlation of the first channel features includes: Based on the correlation between the first channel features and the second channel features, L undetermined similarities are obtained between the first vector and the second vector.

3. The method according to claim 1, characterized in that, The first vector includes I first sub-vectors, each of which includes N first channel features. The second vector includes J second sub-vectors, each of which includes M second channel features. The initial retrieval model obtains L undetermined similarities between the first vector and the second vector based on the correlation between each first channel feature and at least two second channel features, including: For the first target sub-vector and the second target sub-vector, the initial retrieval model obtains the undetermined similarity between the first target sub-vector and the second target sub-vector based on the correlation between the N first channel features included in the first target sub-vector and the M second channel features included in the second target sub-vector; the first target sub-vector is one of I first sub-vectors, and the second target sub-vector is one of J second sub-vectors; I first sub-vectors are used as the target first sub-vectors, and J second sub-vectors are used as the target second sub-vectors, to obtain I*J undetermined similarities between the first sub-vectors and the second sub-vectors; Based on I*J undetermined similarities between the first sub-vector and the second sub-vector, L undetermined similarities between the first vector and the second vector are determined.

4. The method according to claim 3, characterized in that, The step of determining L undetermined similarities between the first sub-vector and the second sub-vector based on I*J undetermined similarities includes: For the i-th first sub-vector among I first sub-vectors, the largest similarity among the J undetermined similarities between the i-th first sub-vector and the second sub-vector is taken as the similarity between the i-th first sub-vector and the second sub-vector; i is a positive integer less than I; Based on the similarity between the first sub-vector and the second sub-vector, the undetermined similarity between the first vector and the second vector is determined.

5. The method according to claim 4, characterized in that, The method further includes: For the j-th second sub-vector among J second sub-vectors, the largest similarity among the I undetermined similarities between the j-th second sub-vector and the first sub-vector is taken as the similarity between the j-th second sub-vector and the first sub-vector; j is a positive integer less than J; The step of determining the undetermined similarity between the first vector and the second vector based on the similarity between each of the first sub-vectors and the second sub-vector includes: The similarity between the first vector and the second vector is determined based on the similarity between I first sub-vectors and the second sub-vectors respectively, and the similarity between J second sub-vectors and the first sub-vectors.

6. The method according to claim 5, characterized in that, The method further includes: Obtain the first weights of I first sub-vectors in the first vector, and the second weights of J second sub-vectors in the second vector; The step of determining the similarity between the first vector and the second vector based on the similarity between I first sub-vectors and the second sub-vectors respectively, and the similarity between J second sub-vectors and the first sub-vectors, includes: The similarity between the first vector and the second vector is determined based on the similarity between the I first sub-vectors and the second sub-vectors, the first weight, the similarity between the J second sub-vectors and the first sub-vectors, and the second weight.

7. The method according to claim 1, characterized in that, For the p-th training of the initial retrieval model, where p is a positive integer; the method further includes: Obtain B pairs of sample pairs required for training the initial retrieval model for the pth time, and B similarities between the first vector and the second vector obtained by the initial retrieval model for the sample pairs; B is an integer greater than 1; The step of training the parameters of the initial retrieval model based on the similarity between the first vector and the second vector, and the sample labels, to obtain the retrieval model includes: Determine the similarity between the first sample pair and other sample pairs; the first sample pair is a pair of positive sample pairs, the positive sample pair is a pair of sample pairs in the sample pair in which the sample label represents the correlation between the first content and the second content, and the other sample pairs are the sample pairs in the sample pairs in the sample pair excluding the first sample pair; If the similarity between the first sample pair and the other sample pairs meets the similarity threshold, then the sample pairs that meet the similarity threshold are added to the positive sample set of the first sample pair; if the similarity between the first sample pair and the other sample pairs does not meet the similarity threshold, then the sample pairs that do not meet the similarity threshold are added to the negative sample set of the first sample pair. Using B pairs of samples as the first sample pairs, the parameters of the initial retrieval model are trained based on the training direction of maximizing the similarity of sample pairs in the positive sample set of the first sample pairs and minimizing the similarity of sample pairs in the negative sample set of the first sample pairs, to obtain the retrieval model. The positive sample set of the first sample pairs includes the first sample pairs.

8. The method according to claim 7, characterized in that, If the data type of the first content is text and the data type of the second content is video, determining the sample similarity between the first sample pair and other sample pairs includes: Obtain a first text vector and a second text vector, wherein the first text vector is the first vector of the first content in the first sample pair, the second text vector is the first vector of the first content in the second sample pair, and the second sample pair is one of the other sample pairs; Based on the first text vector and the second text vector, the similarity between the first text vector and the second text vector is determined, and the similarity between the first text vector and the second text vector is determined as the sample similarity between the first sample pair and the second sample pair; The sample pairs in the other samples are used as the second sample pairs to determine the sample similarity between the first sample pair and the other sample pairs.

9. The method according to any one of claims 1-8, characterized in that, The method further includes: Obtain the search content and the content to be displayed, wherein the search content is the first content and the content to be displayed is the second content; or, the search content is the second content and the content to be displayed is the first content. The search content and the content to be displayed are input into the search model to obtain the similarity between the search content and the content to be displayed; If the similarity between the searched content and the content to be displayed meets the threshold condition, the content to be displayed will be used as the search result of the searched content and displayed.

10. The method according to any one of claims 1-8, characterized in that, The first content and the second content have different data types.

11. A training device for a retrieval model, characterized in that, The device includes: a first acquisition unit, a second acquisition unit, a first determination unit, a second determination unit, and a training unit; The first acquisition unit is used to acquire sample pairs with sample tags, the sample pairs including first content and second content, and the sample tags are used to identify whether the first content and the second content are related; The second acquisition unit is used to acquire a first vector for identifying the first content and a second vector for identifying the second content; the first vector includes N first channel features and the second vector includes M second channel features, where N is an integer greater than 1 and M is an integer greater than 1. The first determining unit is used to input the first vector and the second vector into an initial retrieval model, and obtain L undetermined similarities between the first vector and the second vector based on the correlation between each of the first channel features and at least two of the second channel features; where L is an integer greater than 1. The second determining unit is used to take the largest undetermined similarity among the L undetermined similarities as the similarity between the first vector and the second vector; The training unit is used to train the parameters of the initial retrieval model based on the similarity between the first vector and the second vector, and the sample labels, to obtain the retrieval model; Wherein, if N equals M, the first determining unit is specifically used for: Using the initial retrieval model, the M second channel features in the second vector are rolled K times according to a preset step size to obtain K second rolling vectors. The K second rolling vectors and the second vector are used as a second vector set. The second rolling vector includes M second channel features, and K is a positive integer. Based on the first vector and the vectors in the second vector set, the correlation of the first channel feature is determined according to the position of the channel feature. The correlation of the nth first channel feature in the first vector is determined based on the nth first channel feature and the nth second channel feature of each vector in the second vector set, where n is an integer less than N. Based on the correlation of the first channel features, L undetermined similarities are obtained between the first vector and the second vector.

12. The apparatus according to claim 11, characterized in that, After obtaining the K second rolling vectors, the first determining unit is further configured to: Using the initial retrieval model, the N first channel features in the first vector are rolled K times according to the preset step size to obtain K first rolling vectors. The K first rolling vectors and the first vector are used as a first vector set, and the first rolling vector includes N first channel features. Based on the vectors in the first vector set and the second vector, the correlation for the second channel feature is determined according to the position of the channel feature. The correlation of the m-th second channel feature in the second vector is determined based on the m-th second channel feature and the m-th first channel feature of each vector in the first vector set, where m is an integer less than M. The step of obtaining L undetermined similarities between the first vector and the second vector based on the correlation of the first channel features includes: Based on the correlation between the first channel features and the second channel features, L undetermined similarities are obtained between the first vector and the second vector.

13. A computer device, characterized in that, The computer device includes a processor and memory: The memory is used to store computer programs and to transfer the computer programs to the processor; The processor is configured to execute the method described in any one of claims 1-10 according to instructions in the computer program.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program for performing the method according to any one of claims 1-10.

15. A computer program product comprising a computer program, characterized in that, When it is run on a computer device, it causes the computer device to perform the method described in any one of claims 1-10.

Images