Search data processing method and device, computer device and storage medium

Abstract

The application relates to a search data processing method and device, computer equipment, a storage medium and a computer program product. The application relates to artificial intelligence technology. The method comprises the following steps: acquiring a plurality of first training sample pairs from a media search graph; the plurality of first training sample pairs comprise positive node pairs and negative node pairs; sampling the media search graph based on meta-paths corresponding to search words and media data, to obtain sub-graphs corresponding to search word nodes and media nodes in the first training sample pairs; inputting the sub-graphs corresponding to the respective nodes in the first training sample pairs into an initial graph neural network, to obtain initial semantic features of the respective nodes, and to form semantic feature pairs corresponding to the first training sample pairs; and training the initial graph neural network based on the differences between the semantic feature pairs corresponding to the positive node pairs and the negative node pairs, to obtain a target graph neural network. The method can improve feature extraction accuracy and search accuracy.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a search data processing method, apparatus, computer equipment, storage medium, and computer program product. Background Art

[0002] With the development of computer technology, search functions have appeared in various applications, allowing users to search for relevant media data in the application. For example, in video applications, users can search for videos; in social applications, users can search for images and articles; and so on.

[0003] In traditional technology, features are usually extracted directly from search terms or media data to obtain semantic features corresponding to the search terms or media data, and searches are performed based on the semantic features corresponding to the search terms or media data. However, the information contained in the search terms or media data is limited, which can easily lead to inaccurate semantic features and thus inaccurate searches. Summary of the Invention

[0004] Based on this, it is necessary to provide a search data processing method, device, computer equipment, computer-readable storage medium and computer program product that can improve feature extraction accuracy and search accuracy in response to the above technical problems.

[0005] This application provides a search data processing method. The method includes:

[0006] Obtaining a media search graph; the media search graph includes a search term node corresponding to a search term, a media node corresponding to media data, and an association node corresponding to association data, wherein the association data includes data associated with at least one of the search term and the media data;

[0007] Acquire a plurality of first training sample pairs from the media search graph; the plurality of first training sample pairs include positive node pairs and negative node pairs, the positive node pairs include search term nodes and media nodes connected to each other in the media search graph, and the negative node pairs include search term nodes and media nodes randomly combined in the media search graph;

[0008] Sampling the media search graph based on the meta-paths corresponding to the search term and the media data, respectively, to obtain sampling subgraphs corresponding to the search term node and the media node in the first training sample pair, respectively; the meta-path being a sampling path starting from the search term node or the media node in the media search graph;

[0009] Inputting the sampling subgraphs corresponding to the search term node and the media node in the first training sample pair into the initial graph neural network, obtaining the initial semantic features of the search term node and the media node in the first training sample pair, and forming the semantic feature pairs corresponding to the first training sample pair;

[0010] Based on the differences between the semantic feature pairs corresponding to the positive node pairs and the negative node pairs, the initial graph neural network is trained to obtain a target graph neural network; the target graph neural network is used to determine the target semantic features corresponding to the search term nodes or media nodes.

[0011] The present application also provides a search data processing device. The device includes:

[0012] a media search graph acquisition module, configured to acquire a media search graph; the media search graph comprising a search term node corresponding to a search term, a media node corresponding to media data, and an association node corresponding to association data, wherein the association data comprises data associated with at least one of the search term and the media data;

[0013] a training sample pair acquisition module, configured to acquire a plurality of first training sample pairs from the media search graph; the plurality of first training sample pairs comprising positive node pairs and negative node pairs, the positive node pairs comprising search term nodes and media nodes interconnected in the media search graph, and the negative node pairs comprising search term nodes and media nodes randomly combined in the media search graph;

[0014] a media search graph sampling module, configured to sample the media search graph based on meta-paths corresponding to the search term and the media data, respectively, to obtain sampling subgraphs corresponding to the search term node and the media node in the first training sample pair, respectively; the meta-path being a sampling path starting from the search term node or the media node in the media search graph;

[0015] an initial semantic feature determination module, configured to input the sampling subgraphs corresponding to the search term node and the media node in the first training sample pair into an initial graph neural network, obtain initial semantic features of the search term node and the media node in the first training sample pair, and form a semantic feature pair corresponding to the first training sample pair;

[0016] A network training module is used to train the initial graph neural network based on the differences between the semantic feature pairs corresponding to the positive node pairs and the negative node pairs to obtain a target graph neural network; the target graph neural network is used to determine the target semantic features corresponding to the search term nodes or media nodes.

[0017] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned search data processing method when executing the computer program.

[0018] A computer-readable storage medium stores a computer program, which implements the steps of the above-mentioned search data processing method when executed by a processor.

[0019] A computer program product includes a computer program, wherein when the computer program is executed by a processor, the steps of the search data processing method are implemented.

[0020] The above-mentioned search data processing method, apparatus, computer equipment, storage medium and computer program product obtain a media search graph, wherein the media search graph includes search term nodes corresponding to search terms, media nodes corresponding to media data, and association nodes corresponding to association data, wherein the association data includes data associated with at least one of the search terms and media data, and obtains a plurality of first training sample pairs from the media search graph, wherein the plurality of first training sample pairs include positive node pairs and negative node pairs, wherein the positive node pairs include search term nodes and media nodes interconnected in the media search graph, and the negative node pairs include search term nodes and media nodes randomly combined in the media search graph, and the media search graph is processed based on the meta-paths corresponding to the search terms and media data respectively. Sampling is performed to obtain sampling subgraphs corresponding to the search term node and media node in the first training sample pair. The metapath is a sampling path starting from the search term node or media node in the media search graph. The sampling subgraphs corresponding to the search term node and media node in the first training sample pair are input into the initial graph neural network to obtain initial semantic features for the search term node and media node in the first training sample pair, forming semantic feature pairs corresponding to the first training sample pair. The initial graph neural network is trained based on the differences between the semantic feature pairs corresponding to the positive node pair and the negative node pair to obtain a target graph neural network. The target graph neural network is used to determine target semantic features corresponding to the search term node or media node. In this way, the media search graph includes not only search term nodes and media nodes, but also associated nodes. The media search graph contains rich information. Based on the metapath corresponding to the corresponding node, the media search graph is sampled to obtain a sampling subgraph corresponding to the corresponding node. The sampling subgraph also contains rich information, which helps improve the accuracy of subsequent feature extraction. The sampling subgraph corresponding to the node is input into the trained target graph neural network for feature extraction, and the target semantic feature corresponding to the node is obtained. The target semantic feature incorporates the node information corresponding to each node in the sampling subgraph, has high accuracy, and can also effectively improve the accuracy of subsequent searches. Moreover, when training the graph neural network, the positive node pairs include associated search term nodes and media nodes, and the negative node pairs include unrelated search term nodes and media nodes. Based on the differences between the semantic feature pairs corresponding to the positive node pairs and the negative node pairs, the graph neural network is trained. This can improve the accuracy of the graph neural network in feature extraction, so that the graph neural network outputs similar semantic features for associated search term nodes and media nodes, and outputs dissimilar semantic features for unrelated search term nodes and media nodes, thereby effectively improving the accuracy of subsequent searches based on semantic features.

[0021] This application provides a search data processing method. The method includes:

[0022] Obtaining a sampling subgraph corresponding to a target node; the target node includes at least one of a search term node and a media node, and the sampling subgraph corresponding to the target node is sampled from a media search graph;

[0023] Inputting the sampling subgraph corresponding to the target node into the target graph neural network to obtain the target semantic feature corresponding to the target node; the target semantic feature is used for data search;

[0024] The training process of the target graph neural network includes the following steps: obtaining the media search graph; the media search graph includes search term nodes corresponding to the search term, media nodes corresponding to the media data, and association nodes corresponding to the association data, and the association data includes data associated with at least one of the search term and the media data; obtaining a plurality of first training sample pairs from the media search graph; the plurality of first training sample pairs include positive node pairs and negative node pairs, the positive node pairs include search term nodes and media nodes connected to each other in the media search graph, and the negative node pairs include search term nodes and media nodes randomly combined in the media search graph; based on the metadata corresponding to the search term and the media data respectively path, sampling the media search graph to obtain sampling subgraphs corresponding to the search term nodes and media nodes in the first training sample pair respectively; the meta-path is a sampling path starting from the search term node or media node in the media search graph; the sampling subgraphs corresponding to the search term nodes and media nodes in the first training sample pair are input into the initial graph neural network to obtain the initial semantic features of the search term nodes and media nodes in the first training sample pair, which constitute the semantic feature pairs corresponding to the first training sample pair; based on the difference between the semantic feature pairs corresponding to the positive node pair and the negative node pair, the initial graph neural network is trained to obtain the target graph neural network.

[0025] In one embodiment, the method further comprises:

[0026] Obtain target semantic features corresponding to current search data of a current search object as current search features, and obtain target semantic features corresponding to candidate search data as candidate search features; the current search data is a current search term or current media data, and the candidate search data is a candidate search term or candidate media data; based on the feature similarity between the current search feature and the candidate search feature, determine target search data from multiple candidate search data as the search result corresponding to the current search data.

[0027] In one embodiment, the method further comprises:

[0028] The candidate search features corresponding to each candidate search data are input into the first processing branch of the dual-tower search model to obtain the first search features corresponding to each candidate search data; the current search feature is input into the second processing branch of the dual-tower search model to obtain the second search feature; the first search feature and the second search feature are input into the output layer of the dual-tower search model to obtain the first matching degree between each candidate search data and the current search data; based on the first matching degree, the target search data is determined from the each candidate search data as the search result corresponding to the current search data.

[0029] In one embodiment, the step of inputting the current search feature into the second processing branch of the dual-tower search model to obtain a second search feature includes:

[0030] The target object feature corresponding to the current search object and the current search feature are input into the second processing branch of the dual-tower search model to obtain a second search feature.

[0031] In one embodiment, determining target search data from each candidate search data as the search result corresponding to the current search data based on the first matching degree includes:

[0032] Based on the first matching degree, intermediate search data is determined from the candidate search data, and the target semantic feature corresponding to the intermediate search data is used as the intermediate search feature; the current search feature and the intermediate search feature are input into a single-tower search model to obtain a second matching degree between the intermediate search data and the current search data; based on the second matching degree, target search data is determined from the intermediate search data as the search result corresponding to the current search data.

[0033] In one embodiment, inputting the current search feature and the intermediate search feature into a single-tower search model to obtain a second matching degree between the intermediate search data and the current search data includes:

[0034] The current search feature, the intermediate search feature, and the feature similarity between the current search feature and the intermediate search feature are input into a single-tower search model to obtain a second matching degree between the intermediate search data and the current search data.

[0035] In one embodiment, inputting the current search feature, the intermediate search feature, and the feature similarity between the current search feature and the intermediate search feature into a single-tower search model to obtain a second matching degree between the intermediate search data and the current search data includes:

[0036] The current search feature, the intermediate search feature, the feature similarity between the current search feature and the intermediate search feature, and the feature similarity between the target object feature corresponding to the current search object and the intermediate search feature are input into a single-tower search model to obtain a second matching degree between the intermediate search data and the current search data.

[0037] The present application also provides a search data processing device. The device includes:

[0038] An image acquisition module is configured to acquire a sampling subgraph corresponding to a target node; the target node includes at least one of a search term node and a media node, and the sampling subgraph corresponding to the target node is sampled from a media search graph;

[0039] A feature determination module is used to input the sampling subgraph corresponding to the target node into the target graph neural network to obtain the target semantic feature corresponding to the target node; the target semantic feature is used for data search;

[0040] The training process of the target graph neural network includes the following steps: obtaining the media search graph; the media search graph includes search term nodes corresponding to the search term, media nodes corresponding to the media data, and association nodes corresponding to the association data, and the association data includes data associated with at least one of the search term and the media data; obtaining a plurality of first training sample pairs from the media search graph; the plurality of first training sample pairs include positive node pairs and negative node pairs, the positive node pairs include search term nodes and media nodes connected to each other in the media search graph, and the negative node pairs include search term nodes and media nodes randomly combined in the media search graph; based on the metadata corresponding to the search term and the media data respectively path, sampling the media search graph to obtain sampling subgraphs corresponding to the search term nodes and media nodes in the first training sample pair respectively; the meta-path is a sampling path starting from the search term node or media node in the media search graph; the sampling subgraphs corresponding to the search term nodes and media nodes in the first training sample pair are input into the initial graph neural network to obtain the initial semantic features of the search term nodes and media nodes in the first training sample pair, which constitute the semantic feature pairs corresponding to the first training sample pair; based on the difference between the semantic feature pairs corresponding to the positive node pair and the negative node pair, the initial graph neural network is trained to obtain the target graph neural network.

[0041] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned search data processing method when executing the computer program.

[0042] A computer-readable storage medium stores a computer program, which implements the steps of the above-mentioned search data processing method when executed by a processor.

[0043] A computer program product includes a computer program, wherein when the computer program is executed by a processor, the steps of the search data processing method are implemented.

[0044] The above-mentioned search data processing method, apparatus, computer device, storage medium, and computer program product obtain a sampling subgraph corresponding to a target node. The sampling subgraph corresponding to the target node is sampled from a media search graph. The target node includes at least one of a search term node and a media node. The sampling subgraph corresponding to the target node is input into a target graph neural network to obtain a target semantic feature corresponding to the target node. The target semantic feature is used for data search. The training sample pairs of the graph neural network include positive node pairs and negative node pairs. The positive node pairs include search term nodes and media nodes that are interconnected in the media search graph, and the negative node pairs include search term nodes and media nodes that are randomly combined in the media search graph. The media search graph includes not only search term nodes and media nodes, but also associated nodes. The media search graph contains rich information. Then, based on the meta-path corresponding to the corresponding node, the sampling subgraph corresponding to the corresponding node is sampled from the media search graph. The sampling subgraph also contains rich information, which helps to improve the accuracy of subsequent feature extraction. When training a graph neural network, positive node pairs include associated search term nodes and media nodes, and negative node pairs include unrelated search term nodes and media nodes. Training the graph neural network based on the differences between the semantic feature pairs corresponding to the positive and negative node pairs can improve the accuracy of the graph neural network in feature extraction, so that the graph neural network outputs similar semantic features for associated search term nodes and media nodes, and dissimilar semantic features for unrelated search term nodes and media nodes, thereby effectively improving the accuracy of subsequent searches based on semantic features. Inputting the sampling subgraph corresponding to the node into the trained target graph neural network for feature extraction can obtain the target semantic feature corresponding to the node. The target semantic feature incorporates the node information corresponding to each node in the sampling subgraph, has high accuracy, and can also effectively improve the accuracy of subsequent searches. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A diagram illustrating an application environment of a search data processing method according to an embodiment;

[0046] Figure 2 A flowchart of a search data processing method according to an embodiment is shown;

[0047] Figure 3A This is a schematic diagram of an interface for a list-based search scenario in one embodiment;

[0048] Figure 3B A schematic diagram of an interface for an immersive search scenario in one embodiment;

[0049] Figure 4 is a schematic diagram of a video search graph in one embodiment;

[0050] Figure 5 Schematic diagram of calculating semantic features corresponding to a central node in one embodiment;

[0051] Figure 6 1. A schematic diagram of a process for training a graph neural network in one embodiment;

[0052] Figure 7 is a schematic diagram of a positive sampling sub-image pair and a negative sampling sub-image pair in one embodiment;

[0053] Figure 8 A flowchart of a search data processing method according to another embodiment;

[0054] Figure 9 A schematic diagram of a dual-tower search model in one embodiment;

[0055] Figure 10 A schematic diagram of a flow chart of a single-tower search model in one embodiment;

[0056] Figure 11 A schematic diagram of the architecture of a search data processing method in one embodiment;

[0057] Figure 12 A schematic diagram of a video search process in one embodiment;

[0058] Figure 13 is a structural block diagram of a search data processing device in one embodiment;

[0059] Figure 14 A structural block diagram of a search data processing device in another embodiment;

[0060] Figure 15 A structural block diagram of a search data processing device in another embodiment;

[0061] Figure 16 is a diagram of the internal structure of a computer device in one embodiment;

[0062] Figure 17 FIG. 4 is a diagram showing the internal structure of a computer device in another embodiment. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0064] The solutions provided in the embodiments of this application involve artificial intelligence computer vision technology, machine learning and other technologies, which are specifically illustrated by the following embodiments:

[0065] The search data processing method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other servers. The terminal 102 can be, but is not limited to, various desktop computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented as an independent server or a server cluster consisting of multiple servers or a cloud server.

[0066] Both the terminal and the server can be used independently to execute the search data processing method provided in the embodiments of the present application.

[0067] For example, a server obtains a media search graph and obtains multiple first training sample pairs from the media search graph. The media search graph includes search term nodes corresponding to search terms, media nodes corresponding to media data, and associated nodes corresponding to associated data. The associated data includes data associated with at least one of the search terms and media data. The multiple first training sample pairs include positive node pairs and negative node pairs. The positive node pairs include interconnected search term nodes and media nodes in the media search graph, and the negative node pairs include randomly combined search term nodes and media nodes in the media search graph. The server samples the media search graph based on meta-paths corresponding to the search terms and media data, respectively, to obtain sampled sub-graphs corresponding to the search term nodes and media nodes in the first training sample pairs. The meta-paths are sampled paths starting from the search term nodes or media nodes in the media search graph. The server inputs the sampled sub-graphs corresponding to the search term nodes and media nodes in the training sample pairs into an initial graph neural network, obtains initial semantic features for the search term nodes and media nodes in the first training sample pairs, and forms semantic feature pairs corresponding to the first training sample pairs. The server then trains the initial graph neural network based on the differences between the semantic feature pairs corresponding to the positive node pairs and the negative node pairs, to obtain a target graph neural network. The target graph neural network is used to determine the target semantic features corresponding to the search term node or media node.

[0068] The server obtains a sampled subgraph corresponding to a target node. The target node includes at least one of a search term node and a media node. The sampled subgraph corresponding to the target node is sampled from the media search graph. The server inputs the sampled subgraph corresponding to the target node into the target graph neural network to obtain a target semantic feature corresponding to the target node. The target semantic feature is used for data search.

[0069] The terminal and the server can also be used in conjunction to execute the search data processing method provided in the embodiments of the present application.

[0070] For example, the server obtains a media search graph from the terminal, and the server obtains multiple first training sample pairs from the media search graph. The server samples the media search graph based on the meta-paths corresponding to the search terms and media data, respectively, and obtains sampling subgraphs corresponding to the search term nodes and media nodes in the first training sample pairs. The server inputs the sampling subgraphs corresponding to the search term nodes and media nodes in the first training sample pairs into the initial graph neural network, obtains the initial semantic features of the search term nodes and media nodes in the first training sample pairs, and constitutes the semantic feature pairs corresponding to the first training sample pairs. Based on the differences between the semantic feature pairs corresponding to the positive node pairs and the negative node pairs, the server trains the initial graph neural network to obtain the target graph neural network.

[0071] The terminal sends a search request to the server, the search request carrying a data identifier corresponding to at least one of the search term and the media data. Based on the data identifier, the server determines a sampling subgraph corresponding to the target node from the media search graph. The server inputs the sampling subgraph corresponding to the target node into the target graph neural network to obtain a target semantic feature corresponding to the target node. The server determines a search result based on the target semantic feature corresponding to the target node. The server sends the search result to the terminal.

[0072] In one embodiment, Figure 2 As shown, a search data processing method is provided, which is illustrated by applying the method to a computer device. The computer device can be a terminal or a server. The method can be executed by the terminal or the server itself, or it can be implemented through interaction between the terminal and the server. Figure 2 , the search data processing method includes the following steps:

[0073] Step S202, obtaining a media search graph; the media search graph includes search term nodes corresponding to the search term, media nodes corresponding to the media data, and association nodes corresponding to the associated data, and the associated data includes data associated with at least one of the search term and the media data.

[0074] The media search graph is a heterogeneous graph built based on the search subject's search and consumption behavior for media data. A search subject is an object that triggers search and consumption behavior, namely, the search user. Search and consumption behavior refers to the search subject's behavior toward search results during the search process. For example, a search and consumption behavior might include a search user clicking to play a video or viewing an article.

[0075] The media search graph is a heterogeneous graph consisting of multiple nodes and edges. Nodes in the media search graph include search term nodes corresponding to search terms, media nodes corresponding to media data, and association nodes corresponding to associated data. The connection between search term nodes and media nodes is determined based on search consumption behavior. For example, if a user clicks on media data B in the search results for search term A, a connection exists between the search term node corresponding to search term A and the media node corresponding to media data B, and an edge exists between the search term node and the media node. The connection between search term nodes and association nodes is determined based on the association between search terms and associated data. For example, if search term A involves associated data C, a connection exists between the search term node corresponding to search term A and the association node corresponding to associated data C, and an edge exists between the search term node and the association node. The connection between media nodes and association nodes is determined based on the association between media data and associated data. For example, if media data B involves associated data C, a connection exists between the media node corresponding to media data B and the association node corresponding to associated data C, and an edge exists between the media node and the association node.

[0076] Searches are conducted against a search term to search for media data related to the search term. For example, a user can enter "Movie A" as a search term in a search engine. The search engine will then search for media data related to Movie A and return media data related to Movie A. It is understood that the search term can be text entered directly by the user in the search input box or generated through voice recognition of the user's input.

[0077] Media data includes at least one of text, images, videos, and audio. For example, in a video application, a user can enter a search term to search for related videos; in an audio application, a user can enter a search term to search for related audio; and in an image application, a user can enter a search term to search for related images.

[0078] Related data includes data associated with at least one of the search terms and media data. Related data can include at least one of the associated entities, tags, categories, and publishers. An entity refers to something that exists objectively and can be distinguished. It can be a specific person, event, or object, or a concept, typically referring to a person, place, or institution. It's understood that this term has specific extensions in different fields, such as drug names and disease names in medicine, and IP names, producers, and screenwriters in film and television. Media data has corresponding tags. For example, if a video shows a concert of singer A, and singer A is known to be a superstar or idol, the video could be tagged "Singer A; idol; superstar; concert." Media data also has corresponding categories. For example, articles can be categorized by genre into categories like comedy, food, fashion, travel, entertainment, lifestyle, and games. A publisher refers to the publisher of media data, such as the author of an image.

[0079] Furthermore, nodes in the media search graph may have corresponding node features. Node features can be used to describe the attribute information corresponding to the node, or they can be used to describe the statistical information of the search consumption behavior corresponding to the node. Furthermore, edges in the media search graph may have corresponding connection features. Similarly, connection features can be used to describe the attribute information corresponding to the connection relationship, or they can be used to describe the statistical information of the search consumption behavior corresponding to the connection relationship.

[0080] Specifically, the computer device can obtain a media search graph locally or from other devices, construct training data for a graph neural network for extracting image features based on the media search graph, train the graph neural network based on the training data, and extract image features of subgraphs corresponding to relevant nodes in the media search graph based on the trained graph neural network to generate semantic features corresponding to the nodes. The semantic features can be applied in search tasks to improve search accuracy.

[0081] Step S204, obtaining multiple first training sample pairs from the media search graph; the multiple first training sample pairs include positive node pairs and negative node pairs, the positive node pairs include search term nodes and media nodes interconnected in the media search graph, and the negative node pairs include search term nodes and media nodes randomly combined in the media search graph.

[0082] Among them, the training sample pairs are the training data of the graph neural network. The first training sample pair refers to the node pair used for model training. The first training sample pair includes a positive node pair and a negative node pair. The positive node pair includes the search term nodes and media nodes that are interconnected in the media search graph. The search term nodes and media nodes that are interconnected in the media search graph are determined based on the search consumption behavior of the search object, and can reflect the search intention and consumption preference of the search object. Taking the search term nodes and media nodes that are interconnected in the media search graph as positive node pairs can prompt the network to learn knowledge related to the search intention and consumption preference of the search object. The negative node pair includes randomly combined search term nodes and media nodes in the media search graph. The randomly combined search term nodes and media nodes are irrelevant to the search consumption behavior of the search object. Taking the randomly combined search term nodes and media nodes in the media search graph as negative node pairs can prompt the network to learn to distinguish between positive node pairs and negative node pairs, so that the network can better learn knowledge related to the search intention and consumption preference of the search object.

[0083] Specifically, the computer device can obtain interconnected search term nodes and media nodes from a media search graph as positive node pairs. The media search graph includes multiple search term nodes and multiple media nodes, and ultimately, multiple positive node pairs can be obtained. The computer device can also obtain randomly combined search term nodes and media nodes from the media search graph as negative node pairs. The media search graph includes multiple search term nodes and multiple media nodes, and ultimately, multiple negative node pairs can be obtained. The computer device performs self-supervised learning on a graph neural network based on the positive node pairs and the negative node pairs.

[0084] Step S206: Based on the meta-paths corresponding to the search terms and media data, the media search graph is sampled to obtain sampling sub-graphs corresponding to the search term nodes and media nodes in the first training sample pair; the meta-path is a sampling path starting from the search term node or media node in the media search graph.

[0085] A meta-path is a specific path pattern connecting two objects. It can also be called a meta-path. For example, a meta-path is "search term node -> media data -> search term node." This meta-path represents a search term node connected to another search term node via two hops. Meta-paths are a method for mining similarity relationships between objects. A meta-path is a semantic representation of a search term node or media node in a media search graph and is used for data sampling. A meta-path is a sampling path starting from a search term node or media node in the media search graph, with the search term node or media node in the media search graph as the central node.

[0086] In one embodiment, a meta-path is generally a specific path pattern that connects two objects in sequence through an object. A meta-path generally includes three type flags that are connected in sequence to represent the node type, that is, a meta-path is a path pattern composed of three type flags that are connected in sequence. It can be understood that the three type flags can be the same type flags. For example, the three type flags all represent the search type, and the meta-path is represented as "q->q->q", where q represents the search type. The three type flags can include two type flags. For example, the meta-path is represented as "q->i->q", where i represents the media type; the meta-path is represented as "q->e->q", where e represents the entity type. The three type flags can include three type flags. For example, the meta-path is represented as "q->i->t", where t represents the tag type; and the meta-path is represented as "q->i->e".

[0087] A meta-path is used to sample matching nodes from the media search graph to generate a sampled subgraph. The sampled subgraph is generated by sampling corresponding node paths from the media search graph according to the meta-path's path pattern. That is, the sampled subgraph is generated by sampling nodes connected according to the meta-path's path pattern from the media search graph. The sampled subgraph includes a central node and neighboring nodes. The node type corresponding to the central node belongs to the starting type flag in the meta-path, and the node types corresponding to the neighboring nodes belong to other type flags in the meta-path. In the sampled subgraph, any node path from the central node to a leaf neighbor node is a meta-path. For example, the meta-path is "q->i->q." Any search term node belonging to the search type is retrieved from the media search graph as the central node, media nodes belonging to the media type directly connected to the central node are retrieved from the media search graph as first-order neighbor nodes, and search term nodes belonging to the search type directly connected to the first-order neighbor nodes are retrieved from the media search graph as second-order neighbor nodes. The sampled subgraph is generated based on the central node, first-order neighbor nodes, and second-order neighbor nodes connected according to the meta-path's path pattern.

[0088] There are corresponding meta-paths for search terms and media data. In the meta-path corresponding to the search terms, the node type of the central node is the search type. In the meta-path corresponding to the media data, the node type of the central node is the media type.

[0089] Specifically, the computer device obtains a meta-path corresponding to the search term, samples the media search graph based on the meta-path corresponding to the search term, and obtains a sampled subgraph corresponding to the search term node in the first training sample pair. The computer device obtains a meta-path corresponding to the media data, samples the media search graph based on the meta-path corresponding to the media data, and obtains a sampled subgraph corresponding to the media node in the first training sample pair.

[0090] It can be understood that the meta-path corresponding to the search term includes at least one, and the meta-path corresponding to the media data includes at least one.

[0091] Step S208: Input the sampling subgraphs corresponding to the search term nodes and media nodes in the first training sample pair into the initial graph neural network to obtain the initial semantic features of the search term nodes and media nodes in the first training sample pair, forming the semantic feature pairs corresponding to the first training sample pair.

[0092] A graph neural network is a deep learning-based graph-domain information processing network. It is a connectionist model for learning graphs containing a large number of connections. The input data of the graph neural network is the sampled subgraph corresponding to the node, and the output data is the semantic features of the node. The sampled subgraph corresponding to the node is input into the graph neural network. The graph neural network aggregates the relevant features of the neighboring nodes in the sampled subgraph to the central node, thereby obtaining the semantic features corresponding to the central node. It can be understood that in the sampled subgraph corresponding to a node, the node is the central node in the sampled subgraph.

[0093] The initial graph neural network refers to the graph neural network to be trained. The initial semantic features refer to the semantic features output by the initial graph neural network.

[0094] A first training sample pair includes a pair of search term nodes and media nodes. The search term nodes have corresponding initial semantic features, and the media nodes have corresponding initial semantic features. The initial semantic features corresponding to the search term nodes and media nodes in the first training sample pair constitute a semantic feature pair.

[0095] Specifically, the computer device inputs the sampling subgraph corresponding to the search term node in the first training sample pair into the initial graph neural network to obtain the initial semantic features corresponding to the search term node in the first training sample pair. The computer device also inputs the sampling subgraph corresponding to the media node in the first training sample pair into the initial graph neural network to obtain the initial semantic features corresponding to the media node in the first training sample pair. The initial semantic features corresponding to the search term node and media node in the same training sample pair constitute the semantic feature pair corresponding to the training sample pair.

[0096] In one embodiment, the graph neural network can be a graph convolutional network. A graph convolutional network processes image data through convolution operations. There are many types of graph convolutional networks, such as GraphSage (a spatial domain-based graph convolution algorithm), GAT (Graph Attention Network), and DisenGCN (Disentangled Graph Convolutional Networks).

[0097] Step S210: Based on the differences between the semantic feature pairs corresponding to the positive node pairs and the negative node pairs, the initial graph neural network is trained to obtain a target graph neural network; the target graph neural network is used to determine the target semantic features corresponding to the search term nodes or media nodes.

[0098] The target graph neural network refers to the trained graph neural network, and the target semantic feature refers to the semantic feature output by the target graph neural network.

[0099] Specifically, after obtaining the semantic feature pairs corresponding to each pair of training samples, the computer device trains the initial graph neural network based on the differences between the semantic feature pairs corresponding to the positive node pairs and the negative node pairs to obtain the target graph neural network.

[0100] The computer device generates a network loss based on the difference between the semantic feature pairs corresponding to the positive node pairs and the negative node pairs, respectively, and adjusts the network parameters of the initial graph neural network based on the network loss until the convergence condition is met, thereby obtaining the target graph neural network. For example, if there are overlapping nodes between the matched positive node pairs and the negative node pairs, the feature similarity between the semantic features in the semantic feature pairs corresponding to the positive node pairs is calculated, and the feature similarity between the semantic features in the semantic feature pairs corresponding to the negative node pairs is calculated. The network loss is calculated based on the difference between the feature similarities corresponding to the matched positive node pairs and the negative node pairs, and the network parameters of the initial graph neural network are adjusted based on the network loss so that the difference between the feature similarities corresponding to the matched positive node pairs and the negative node pairs increases, that is, the more similar the semantic features between the search term nodes and the media nodes with edges are, the less similar the semantic features between the search term nodes and the media nodes without edges are.

[0101] The convergence condition refers to the condition for judging whether the network has reached convergence. The convergence condition includes but is not limited to at least one of the following: the network loss is less than a preset loss value, the number of network iterations is greater than a preset number of iterations, or the rate of change of the network loss is less than a preset rate of change.

[0102] For example, a computer device obtains a first training sample pair, inputs the sampling subgraphs corresponding to the search term node and the media node in the first training sample pair into the initial graph neural network, obtains the semantic feature pair corresponding to the first training sample pair, calculates the network loss based on the semantic feature pairs corresponding to the positive node pair and the negative node pair, adjusts the initial graph neural network based on the network loss to obtain an intermediate graph neural network, uses the intermediate graph neural network as the initial graph neural network, obtains a new first training sample pair, inputs the new first training sample pair into the new initial graph neural network to calculate a new network loss, adjusts the new initial graph neural network based on the new network loss to obtain a new intermediate graph neural network, uses the intermediate graph neural network as the initial graph neural network, returns to the step of obtaining the training sample pair, and continues iterative training. If the preset number of iterations is 100, the intermediate graph neural network obtained by the 101st adjustment is obtained as the target graph neural network.

[0103] The target graph neural network is used to determine the target semantic features corresponding to search term nodes or media nodes. The target semantic features corresponding to search term nodes or media nodes can be applied to search scenarios such as q2q (query-to-query), q2i (query-to-item), i2q (item-to-query), and i2i (item-to-item).

[0104] In q2q search scenarios, the query currently entered by the user is semantically expanded or rewritten to explore the potential search needs behind the query currently entered by the user, and subsequently provide the user with richer media data of potential interest. For example, in a q2q search scenario, a sampled subgraph of the search term nodes corresponding to the search term is input into the target graph neural network to obtain the target semantic features corresponding to the search term. Based on the differences between the target semantic features corresponding to search term A and the target semantic features corresponding to each other search term, relevant search terms corresponding to search term A are determined from each other search term, and the relevant search terms are used as the search results for search term A in the q2q scenario.

[0105] In a q2i search scenario, media data semantically related to the user's query is retrieved for the user. For example, in a q2i search scenario, a sampled subgraph of the search term nodes corresponding to the search term is input into the target graph neural network to obtain the target semantic features corresponding to the search term. A sampled subgraph of the media nodes corresponding to the media data is input into the target graph neural network to obtain the target semantic features corresponding to the media data. Based on the differences between the target semantic features corresponding to search term A and the target semantic features corresponding to each piece of media data, the relevant media data corresponding to search term A is determined from each piece of media data and used as the search result for search term A in the q2i scenario.

[0106] In the i2q search scenario, queries semantically related to the media data currently consumed by users are retrieved, guiding them to conduct secondary searches to meet their potential needs. For example, in the i2q search scenario, a sampling subgraph of media nodes corresponding to the media data is input into the target graph neural network to obtain the target semantic features corresponding to the media data. A sampling subgraph of search term nodes corresponding to the search term is input into the target graph neural network to obtain the target semantic features corresponding to the search term. Based on the differences between the target semantic features corresponding to media data A and the target semantic features corresponding to each search term, relevant search terms corresponding to media data A are determined from each search term, and the relevant search terms are used as the search results for media data A in the i2q scenario.

[0107] In the i2i search scenario, semantically related media data is recommended to users based on their consumption preferences for the currently recalled media data list. For example, in the i2i search scenario, a sampled subgraph of media nodes corresponding to the media data is input into the target graph neural network to obtain the target semantic features corresponding to the media data. Based on the differences between the target semantic features corresponding to media data A and the target semantic features corresponding to each other media data, relevant media data corresponding to media data A is determined from each other media data and used as the search result for media data A in the i2i scenario.

[0108] In the above-mentioned search data processing method, the media search graph includes not only search term nodes and media nodes, but also associated nodes. The media search graph contains rich information. Then, based on the meta-path corresponding to the corresponding node, a sampled subgraph corresponding to the corresponding node is sampled from the media search graph. The sampled subgraph also contains rich information, which helps to improve the accuracy of subsequent feature extraction. The sampled subgraph corresponding to the node is input into the trained target graph neural network for feature extraction, and the target semantic feature corresponding to the node can be obtained. The target semantic feature is integrated with the node information corresponding to each node in the sampled subgraph, has high accuracy, and can also effectively improve the accuracy of subsequent searches. In addition, when training the graph neural network, positive node pairs include associated search term nodes and media nodes, and negative node pairs include unrelated search term nodes and media nodes. Based on the differences between the semantic feature pairs corresponding to the positive node pairs and the negative node pairs, the graph neural network is trained to improve the accuracy of the graph neural network in feature extraction, so that the graph neural network outputs similar semantic features for associated search term nodes and media nodes, and outputs dissimilar semantic features for unrelated search term nodes and media nodes, thereby effectively improving the accuracy of subsequent searches based on semantic features.

[0109] In one embodiment, obtaining a media search graph includes:

[0110] Obtain historical search information corresponding to multiple search objects; the historical search information includes historical search terms and positive media data corresponding to the historical search terms, and the positive media data is the media data of the positive feedback of the search objects; generate object nodes corresponding to each search object, search term nodes corresponding to each historical search term, and media nodes corresponding to each positive media data, establish a connection relationship between the object node and the corresponding search term node, establish a connection relationship between the object node and the corresponding media node, and establish a connection relationship between the search term node and the corresponding media node; obtain at least one associated data corresponding to the historical search term and the positive media data; at least one associated data includes an associated entity, an associated tag, an associated category or an associated publishing object; generate an associated node corresponding to each associated data, establish a connection relationship between the search term node and the corresponding associated node, and establish a connection relationship between the media node and the corresponding associated node; obtain node features corresponding to each node and connection features corresponding to each connection relationship; generate a media search graph based on each node, the node features corresponding to the nodes, the connection relationship between the nodes, and the connection features corresponding to the connection relationship.

[0111] The search object refers to the search user. Historical search information is search information determined based on the search user's historical search consumption behavior and is used to reflect the search user's historical search situation. Historical search information includes historical search terms and positive media data corresponding to the historical search terms. Historical search terms refer to search terms used by the search user in past searches. Positive media data is media data that is positively fed back by the search object, that is, positive media data refers to media data consumed by the user in the media data search results. For example, for the media data search results corresponding to the search term A entered by the user, the positive media data is the media data clicked by the user.

[0112] In one embodiment, to enrich the data in the media search graph, search logs corresponding to the search object in at least two search scenarios are obtained, and historical search information is mined from the search logs. Furthermore, the search logs can be cleaned to obtain historical search information. For example, search logs missing key information (key information includes the search object, historical search terms, and forward media data) can be cleaned; data with abnormal consumption duration can be cleaned; and so on.

[0113] In one embodiment, in order to better explore the user's real interests, the positive media data includes media data with a consumption time greater than a duration threshold. The longer the user consumes the media data, the more interested the user is in the media data. Furthermore, the positive media data includes at least one of media data with a consumption time greater than a first duration threshold in a list search scenario and media data with a consumption time greater than a second duration threshold in an immersive search scenario, and the first duration threshold is less than the second duration threshold. For example, for a video, a video that meets one of the following two conditions is considered to be positively consumed: the user clicks on a video in the search list page, and the playback time meets a certain duration threshold α, and the user plays a video in the search immersive page to meet a certain duration threshold β, where α<β. A list search scenario refers to a search scenario in which multiple search results in a search result list are displayed synchronously in a list format. For example, refer to Figure 3A , is the video search result list page obtained after the user searches for "Animation A". The user views the search results by swiping up or down the search result list, clicks on a video to enter the playback page for consumption, and returns from the playback page to the list page to continue viewing other search results in the list page. Immersive search scenarios refer to search scenarios where personalized searches are performed based on the search results consumed by the user. For example, refer to Figure 3B , is the video playback page that the user gets when clicking on a video search result of a query. When the user clicks to enter the playback page, an "immersive page search" will be initiated. This scenario is a personalized search scenario. The user can view other immersive search results of personalized search by swiping up or down the playback page. Further, Figure 3B The 302 is based on Figure 3B The video played in the video is the related query recalled by the user, which guides the user to conduct a secondary search to meet his potential needs.

[0114] Historical search terms and forward media data each have corresponding associated data. Associated data includes at least one of associated entities, associated tags, associated categories, and associated publishing objects. In one embodiment, to improve data accuracy, associated data can remove entities with inaccurate entity identification to prevent the introduction of erroneous information, remove rare categories to prevent unreliable category information, and normalize publishing objects, uniting multiple publishing accounts corresponding to the same user into a single account.

[0115] It can be understood that the media search graph includes not only the connections between nodes but also node features corresponding to the nodes and connection features corresponding to the connection relationships. Node features include at least one of the corresponding attribute features and statistical features of the nodes, while connection features include at least one of the corresponding attribute features and statistical features of the connection relationships. Attribute features reflect attribute information, while statistical features reflect statistical information about search and consumption behavior.

[0116] Taking the short video node as an example, its node characteristics can be shown in Table 1.

[0117] Table 1

[0118]

[0119] Specifically, the computer device obtains historical search information corresponding to multiple search objects, uses the search objects, historical search terms, and positive media data as basic composition elements, generates object nodes corresponding to each search object, search term nodes corresponding to each historical search term, and media nodes corresponding to each positive media data, establishes a connection relationship between the object node and the corresponding search term node, establishes a connection relationship between the object node and the corresponding media node, and establishes a connection relationship between the search term node and the corresponding media node. A media search graph is established based on the connection relationship between the object node, the search term node, the media node, and the corresponding node. For example, for the historical search term qy of the search object ux and the media data iz that is positively consumed in the search results of the historical search term qy, nodes corresponding to ux, qy, and iz are constructed, and the ux, qy, and iz nodes are added to the composition, with edges between each of them.

[0120] Furthermore, associated data corresponding to historical search terms and forward media data can be used as additional composition elements. The computer device obtains at least one type of associated data corresponding to each of the historical search terms and forward media data, generates associated nodes corresponding to each associated data, establishes a connection relationship between the search term node and the corresponding associated node, and establishes a connection relationship between the media node and the corresponding associated node. The connection relationship between the associated node and the associated node is added to the media search graph.

[0121] Furthermore, each element in the media search graph may also have corresponding element features. The computer device obtains the node features corresponding to each node and the connection features corresponding to each connection relationship, adds the node features and the connection features to the media search graph, and uses the node features as node information and the connection features as connection relationship information.

[0122] For example, reference Figure 4, is a schematic diagram of a video search graph. The video search graph includes video nodes representing videos, user nodes representing search users, search term nodes representing search terms, author nodes representing video authors, category nodes representing video domain classifications, entity nodes representing entities in videos or search terms, and tag nodes representing video tags. The video node is a media node, the user node is an object node, and the author node, category node, entity node, and tag node are different types of association nodes.

[0123] In the above embodiment, the core nodes of the media search graph are constructed with the three core elements in the search scenario, namely the search object, search term, and media data, and associated nodes associated with the core nodes are introduced into the media search graph, making the entire graph information very rich. Such a media search graph helps to improve the accuracy of subsequent data processing.

[0124] In one embodiment, before generating the associated nodes corresponding to the respective associated data, the search data processing method further includes:

[0125] Obtain supplementary media data and generate a media node corresponding to the supplementary media data; the supplementary media data includes at least one of first media data and second media data, the first media data refers to media data whose media quality is greater than a quality threshold, and the second media data refers to media data whose time interval between a published time and a current time is less than a time interval threshold; obtain at least one associated data corresponding to the supplementary media data.

[0126] Specifically, to ensure the media data coverage of the media search graph, media data that does not appear in historical search information can also be used as supplementary media data, and the supplementary media data can be used as an additional composition element. Supplementary media data refers to additional supplementary media data. Supplementary media data includes at least one of first media data and second media data. First media data refers to media data whose media quality is greater than a quality threshold, that is, first media data refers to high-quality media data, and high-quality media data refers to media data whose quality score reaches a certain threshold. Second media data refers to media data whose time interval between the published time and the current time is less than a time interval threshold, that is, second media data refers to high-relevance media data, and high-relevance media data refers to media data whose publication time is within a short time interval from the current moment, for example, within a week. The computer device obtains the supplementary media data, generates a media node corresponding to the supplementary media data, and adds the media node corresponding to the supplementary media data to the media search graph. Furthermore, the computer device can also obtain at least one type of associated data corresponding to the supplementary media data, add the associated node corresponding to the associated data of the supplementary media data to the media search graph, and add the connection relationship between the supplementary media data and the nodes corresponding to its associated data to the media search graph.

[0127] In the above embodiment, adding high-quality or high-time media data to the media search graph can ensure the media data coverage of the media search graph, making the entire graph very information-rich. Such a media search graph helps to improve the accuracy of subsequent data processing.

[0128] In one embodiment, the search data processing method further includes:

[0129] Obtain a rewritten search term corresponding to a historical search term; when the search time interval between the historical search term and the corresponding rewritten search term is less than a time interval threshold, and the similarity between the historical search term and the corresponding rewritten search term is greater than a similarity threshold, generate a search term node corresponding to the rewritten search term; establish a connection relationship between the search term node corresponding to the historical search term and the search term node corresponding to the rewritten search term.

[0130] Rewriting a search term refers to a search term that a user rewrites during the search process. For example, a user enters search term A and, if dissatisfied with the search results, changes the search term and searches again. The changed search term, search term B, becomes the rewritten search term for search term A.

[0131] The similarity between different search terms is used to represent the degree of similarity between them. The greater the similarity, the more similar the search terms are. Various similarity calculation algorithms can be used to calculate the similarity between search terms. For example, the character overlap ratio between search terms can be calculated as similarity. Alternatively, the text features of the search terms can be extracted and the cosine distance or Euclidean distance between the text features can be calculated as the distance between the text features.

[0132] Specifically, to further enhance the information richness of the media search graph, rewritten search terms corresponding to historical search terms can be used as additional graph elements. The computer device obtains the rewritten search terms corresponding to the historical search terms, generates search term nodes corresponding to the rewritten search terms, and adds them to the media search graph.

[0133] Furthermore, to ensure the accuracy of information in the media search graph, rewritten search terms corresponding to historical search terms are selectively added to the media search graph. The computer device can filter the rewritten search terms, generate search term nodes corresponding to the filtered rewritten search terms, and add them to the media search graph. During the filtering process, if the search time interval between the historical search term and the corresponding rewritten search term is less than a time interval threshold, and the similarity between the historical search term and the corresponding rewritten search term is greater than a similarity threshold, a search term node corresponding to the rewritten search term is generated, a connection relationship is established between the search term node corresponding to the historical search term and the search term node corresponding to the rewritten search term, and the new search term node and connection relationship are added to the media search graph.

[0134] In the above embodiment, based on the search term rewriting behavior of the search object, rewritten search terms with high similarity and search time within a certain interval are added to the media search graph, which can ensure the search term coverage of the media search graph and make the entire graph information very rich. Such a media search graph helps to improve the accuracy of subsequent data processing.

[0135] In one embodiment, based on the meta-paths corresponding to the search terms and media data, the media search graph is sampled to obtain sampling subgraphs corresponding to the search term nodes and media nodes in the training sample pairs, including:

[0136] Determine a current search meta-path from at least one meta-path corresponding to a search term; the current search meta-path is a path formed by sequentially connecting type flags corresponding to the search type, the first type, and the second type, respectively; take the current search term node as the search center node, sample at least two nodes of the first type directly connected to the search center node from the media search graph as first-order neighbor nodes corresponding to the search center node, sample at least two nodes of the second type directly connected to the first-order neighbor node from the media search graph as second-order neighbor nodes corresponding to the search center node, and obtain a sampling subgraph corresponding to the search center node under the current search meta-path based on the search center node and the corresponding first and second neighbor nodes;

[0137] Determine the current media meta-path from at least one meta-path corresponding to the media data; the current media meta-path is a path formed by sequentially connecting type identifiers corresponding to the media type, the third type, and the fourth type; take the current media node as the media center node, sample at least two nodes of the third type directly connected to the media center node from the search media graph as first-order neighbor nodes corresponding to the media center node, sample at least two nodes of the fourth type directly connected to the first-order neighbor node from the search media graph as second-order neighbor nodes corresponding to the media center node, and based on the media center node and the corresponding first neighbor node and second neighbor node, obtain a sampling sub-graph corresponding to the media center node under the current media meta-path.

[0138] The meta-path is a path pattern consisting of three sequentially connected type flags. The search meta-path refers to the meta-path corresponding to the search term. The current search meta-path refers to the search meta-path currently being processed, and can be any meta-path in at least one meta-path corresponding to the search term. The current search meta-path includes sequentially connected type flags representing the search type, the first type, and the second type. It is understood that the first type and the second type can be the same type flag or different type flags, and the first type and the second type can be search types or other types.

[0139] Similarly, a media meta-path refers to the meta-path corresponding to the media data. The current media meta-path refers to the media meta-path currently being processed, and can be any meta-path from at least one meta-path corresponding to the media data. The current media meta-path includes type flags representing the media type, the third type, and the fourth type, connected in sequence. The third and fourth types can have the same type flag or different type flags, and can be media types or other types.

[0140] In one embodiment, each meta-path corresponding to the search term may be as shown in Table 2. Each meta-path corresponding to the media data may be as shown in Table 3.

[0141] Table 2

[0142]

[0143]

[0144] Table 3

[0145]

[0146] The current search term node refers to the search term node currently being processed, which can be any search term node in the media search graph. The current media node refers to the media node currently being processed, which can be any media node in the media search graph.

[0147] Specifically, the computer device samples nodes connected according to the path pattern of the meta-path from the media search graph to generate a sampled subgraph. Taking the current search meta-path as an example, the first type flag in the current search meta-path indicates the search type, and the computer device uses the current search term node as the search center node. In the resulting sampled subgraph, the center node is the current search term node. The second type flag in the current search meta-path indicates the first type, and the computer device samples at least two nodes of the first type directly connected to the search center node from the media search graph as the first-order neighbor nodes corresponding to the search center node. In the resulting sampled subgraph, the search center node and the first-order neighbor nodes are connected. The third type flag in the current search meta-path indicates the second type, and the computer device samples at least two nodes of the second type directly connected to the first-order neighbor nodes from the media search graph as the second-order neighbor nodes corresponding to the search center node. Each first-order neighbor node has its own corresponding second-order neighbor node. In the resulting sampled subgraph, the first-order neighbor nodes and the corresponding second-order neighbor nodes are connected. Finally, the computer device constructs a sampled subgraph corresponding to the search center node under the current search meta-path based on the search center node and its corresponding first and second neighbor nodes.

[0148] Taking the current media meta-path as an example, the first type flag in the current media meta-path indicates the media type, and the computer device uses the current media node as the media center node. In the resulting sampling subgraph, the center node is the current media node. The second type flag in the current media meta-path indicates the third type. The computer device samples at least two nodes of the third type that are directly connected to the media center node from the media search graph as the first-order neighbor nodes of the media center node. In the resulting sampling subgraph, the media center node and the first-order neighbor nodes are connected. The third type flag in the current media meta-path indicates the fourth type. The computer device samples at least two nodes of the fourth type that are directly connected to the first-order neighbor nodes from the media search graph as the second-order neighbor nodes of the media center node. Each first-order neighbor node has its own corresponding second-order neighbor node. In the resulting sampling subgraph, the first-order neighbor nodes and the corresponding second-order neighbor nodes are connected. Finally, the computer device constructs a sampling subgraph corresponding to the media center node under the current media meta-path based on the media center node and its corresponding first and second neighbor nodes.

[0149] In the above embodiment, nodes connected according to the path pattern of the meta-path are sampled from the media search graph to generate a sampling subgraph. The sampling subgraph has multiple first-order neighbor nodes and second-order neighbor nodes, which can improve the information richness of the sampling subgraph, and each neighbor node helps to better reflect the characteristics of the central node from different angles. Such a sampling subgraph can contain more accurate characteristics of the central node, which helps to improve the accuracy of subsequent data processing.

[0150] In one embodiment, the sampling subgraphs corresponding to the search term node and the media node in the first training sample pair are input into the initial graph neural network to obtain the initial semantic features of the search term node and the media node in the first training sample pair, including:

[0151] The sampling subgraphs of at least two semantic perspectives corresponding to the current node are input into the initial graph neural network to obtain the subgraph features of each sampling subgraph corresponding to the current node; the current node is the search term node or media node in the first training sample pair, and different meta-paths correspond to different semantic perspectives; the subgraph features corresponding to the current node are fused to obtain the initial semantic features corresponding to the current node.

[0152] Different meta-paths correspond to different path patterns, and different path patterns mine user search needs from different semantic perspectives. For example, referring to Table 2, the seven meta-paths in Table 2 correspond to different semantic perspectives.

[0153] Specifically, considering the complexity and diversity of node types in the media search graph, and to improve network robustness, at least two meta-paths are designed for search term nodes and media nodes, respectively. Different meta-paths correspond to different semantic perspectives, and subgraph sampling around the central node can be performed along different meta-path directions. Ultimately, at least two sampled subgraphs can be sampled for both search term nodes and media nodes. It is understood that when extracting image features from the sampled subgraphs, the feature extraction method for the sampled subgraphs corresponding to search term nodes and media nodes is the same. The feature extraction process is illustrated using the current node as an example, where the computer device inputs the sampled subgraph corresponding to the current node into the initial graph neural network. The initial graph neural network aggregates the relevant node information of each neighboring node in the sampled subgraph onto the central node, thereby obtaining subgraph features for the sampled subgraph corresponding to the current node. The sampled subgraph corresponding to the current node includes sampled subgraphs from at least two semantic perspectives. After processing by the initial graph neural network, the subgraph features of each sampled subgraph corresponding to the current node are ultimately obtained. Furthermore, the computer device fuses the subgraph features of each sampled subgraph corresponding to the current node to ultimately obtain the initial semantic features corresponding to the current node. The fusion method can be summation, averaging or attention mechanism.

[0154] It can be understood that the fusion operation can be performed inside or outside the graph neural network.

[0155] In the above embodiment, the initial semantic feature corresponding to the current node is obtained by fusing the subgraph features of the sampled subgraphs from different semantic perspectives of the current node. It contains richer information, can express the semantic information of the current node more accurately, has higher accuracy, and also helps to improve the accuracy of the search.

[0156] In one embodiment, the sampling subgraph corresponding to a node includes the node and its first-order neighbor nodes and second-order neighbor nodes. The sampling subgraphs corresponding to at least two semantic perspectives of the current node are input into the initial graph neural network to obtain the subgraph features of each sampling subgraph corresponding to the current node, including:

[0157] Through the initial graph neural network, the node features corresponding to the second-order neighbor nodes in the current sampling subgraph corresponding to the current node, as well as the connection features between the second-order neighbor nodes and the first-order neighbor nodes are aggregated to the first-order neighbor nodes to obtain the second-order aggregate features corresponding to the first-order neighbor nodes; through the initial graph neural network, the node features corresponding to the first-order neighbor nodes, the second-order aggregate features, and the connection features between the first-order neighbor nodes and the current node are aggregated to the current node to obtain the first-order aggregate features corresponding to the current node; through the initial graph neural network, based on the node features and first-order aggregate features corresponding to the current node, the subgraph features of the current sampling subgraph corresponding to the current node are obtained.

[0158] The current sampling subgraph refers to the sampling subgraph currently being processed, and may be any one of the sampling subgraphs of each semantic perspective corresponding to the current node.

[0159] Specifically, the graph neural network performs the same feature extraction operations on each sampled subgraph, and the feature extraction operations of the graph neural network are performed in reverse along the meta-path. Taking the current sampled subgraph corresponding to the current node as an example, the computer device inputs the current sampled subgraph into the initial graph neural network. In the initial graph neural network, the node features corresponding to the second-order neighbor nodes in the current sampled subgraph, as well as the connection features between the second-order neighbor nodes and the first-order neighbor nodes, are first aggregated to the first-order neighbor nodes to obtain the second-order aggregate features corresponding to the first-order neighbor nodes. Then, the node features, second-order aggregate features, and connection features between the first-order neighbor nodes and the current node are aggregated to the current node to obtain the first-order aggregate features corresponding to the current node. Finally, the node features and first-order aggregate features corresponding to the current node are fused to obtain the subgraph features of the current sampled subgraph corresponding to the current node.

[0160] It is understood that when performing aggregation, the individual features are fused to obtain the aggregated features. For example, the aggregated features are obtained by weighted summing of the individual features, where the weights are determined by network parameters.

[0161] For example, reference Figure 5 Taking qie as an example, first aggregate the relevant features of the outermost e node (i.e., the second-order neighbor node) of the sampled subgraph to the corresponding i node (i.e., the first-order neighbor node), and then aggregate the relevant features of the i node to the central node q, thereby obtaining the subgraph features of the central node q from the perspective of qie, that is, the subgraph representation of the central node q from the perspective of qie Taking quq as an example, firstly, the relevant features of the outermost q node (i.e., the second-order neighbor node) of the sampled subgraph are aggregated to the corresponding u node (i.e., the first-order neighbor node), and then the relevant features of the u node are aggregated to the central node q, thereby obtaining the subgraph features of the central node q from the perspective of qie, that is, the subgraph representation of the central node q from the perspective of qie Furthermore, feature extraction is performed on the sampled subgraphs of the same central node q under multiple semantic perspectives to obtain the subgraph features of the central node q under multiple semantic perspectives, that is, the multi-perspective semantic representation of the central node q. After fusing the subgraph features of the central node q under each semantic perspective, the final semantic feature h of the central node q is obtained. q .

[0162] In the above embodiment, the subgraph features of the sampled subgraph are generated by aggregating the relevant information of neighboring nodes in the sampled subgraph onto a central node. During network training, this neighbor information is continuously aggregated and then iteratively updated. As the number of iterations increases, the aggregated information for each node becomes almost global, which helps improve the quality of model training, thereby improving the accuracy of extracting semantic features corresponding to the nodes, and thus improving the accuracy of subsequent searches.

[0163] In one embodiment, reference Figure 6 Based on the differences between the semantic feature pairs corresponding to the positive node pairs and the negative node pairs, the initial graph neural network is trained to obtain the target graph neural network, including:

[0164] Step S602 : obtaining a node loss based on the difference between the semantic feature pairs corresponding to the positive node pairs and the negative node pairs.

[0165] Node loss is used to characterize the difference between the feature correlation between nodes in positive node pairs and the feature correlation between nodes in negative node pairs. The network training goal is to maximize the feature correlation between nodes in positive node pairs and minimize the feature correlation between nodes in negative node pairs. This allows the network to output semantic features that are as similar as possible for matching search terms and media data, and to output semantic features that are as dissimilar as possible for mismatching search terms and media data.

[0166] Specifically, the training data of the network includes a first training sample pair, and the training method based on the first training sample pair is a supervised learning method based on search consumption behavior. In the search scenario, the user's search consumption behavior can be directly used as a supervisory signal. For the first training sample pair, the sampling subgraphs corresponding to the search term nodes and media nodes in the first training sample pair are input into the initial graph neural network to obtain the initial semantic features of the search term nodes and media nodes in the first training sample pair, which constitute the semantic feature pairs corresponding to the first training sample pair. The computer device calculates the node loss based on the difference between the semantic feature pairs corresponding to the positive node pair and the negative node pair.

[0167] Step S604, obtaining multiple second training sample pairs; the multiple second training sample pairs include positive sampling subgraph pairs and negative sampling subgraph pairs, the positive sampling subgraph pairs include sampling subgraphs of different semantic perspectives corresponding to the same node, and the negative sampling subgraph pairs include sampling subgraphs corresponding to different nodes.

[0168] The second training sample pair refers to the sampling subgraph pair used for model training. The second training sample pair includes a positive sampling subgraph pair and a negative sampling subgraph pair. The positive sampling subgraph pair includes sampling subgraphs from different semantic perspectives corresponding to the same node, while the negative sampling subgraph pair includes sampling subgraphs corresponding to different nodes. It can be understood that for the same node, although multiple sampling subgraphs come from different perspectives, their mutual correlation is very high. For subgraph representations between different nodes, even if they come from the same perspective, their mutual correlation is relatively low.

[0169] In step S606, each sampling subgraph in the second training sample pair is input into the initial graph neural network to obtain the subgraph features of each sampling subgraph in the second training sample pair to form a subgraph feature pair corresponding to the second training sample pair.

[0170] Step S608 : Obtaining a viewing angle loss based on the difference between the sub-image feature pairs corresponding to the positive sampling sub-image pair and the negative sampling sub-image pair.

[0171] A second training sample pair includes a pair of sampling subgraphs, the sampling subgraphs have corresponding subgraph features, and the subgraph features corresponding to the respective sampling subgraphs in the second training sample pair constitute a subgraph feature pair.

[0172] Viewpoint loss is used to characterize the difference between the feature correlation between the sampled subimages in a positively sampled subimage pair and the feature correlation between the sampled subimages in a negatively sampled subimage pair. The network training goal is to maximize the feature correlation between the sampled subimages in a positively sampled subimage pair and minimize the feature correlation between the sampled subimages in a negatively sampled subimage pair, so that the subimage features output by the network for different nodes have high discriminability.

[0173] Specifically, the network's training data also includes a second training sample pair, and the training method based on the second training sample pair is a multi-perspective comparative learning method. For the second training sample pair, the computer device inputs each sampled subgraph in the second training sample pair into the initial graph neural network, obtains the subgraph features of each sampled subgraph in the second training sample pair, and forms the subgraph feature pairs corresponding to the second training sample pair. The computer device calculates the perspective loss based on the difference between the subgraph feature pairs corresponding to the positive sampling subgraph pair and the negative sampling subgraph pair.

[0174] It is understood that the perspective loss can include at least one of the perspective loss corresponding to the search term node and the perspective loss corresponding to the media node. Regarding the perspective loss corresponding to the search term, the positive sampling subgraph pair includes sampling subgraphs corresponding to different semantic perspectives for the same search term node, and the negative sampling subgraph pair includes sampling subgraphs corresponding to different search term nodes. Regarding the perspective loss corresponding to the media node, the positive sampling subgraph pair includes sampling subgraphs corresponding to different semantic perspectives for the same media node, and the negative sampling subgraph pair includes sampling subgraphs corresponding to different media nodes.

[0175] Step S610: Train the initial graph neural network based on node loss and view loss to obtain the target graph neural network.

[0176] Specifically, to further improve the training quality of the network, supervised learning of search consumption behavior and multi-view comparative learning methods are combined for joint training. The computer device derives a fused network loss based on node loss and view loss. This fused network loss is then used to train the initial graph neural network to obtain the target graph neural network.

[0177] In the above embodiment, combining node loss and view loss for training enables the network to learn more general knowledge across search scenarios, guiding the model toward diversified optimization and fully exploiting the deep semantic information inherent in the heterogeneous media search graph. The view loss, which embodies multi-view comparative learning, can further mitigate data sparsity and enhance model robustness.

[0178] In one embodiment, the node loss is obtained based on the difference between the semantic feature pairs corresponding to the positive node pairs and the negative node pairs, including:

[0179] Based on the feature similarity between the initial semantic features in the same semantic feature pair, the semantic similarity corresponding to the semantic feature pair is obtained; the semantic similarities corresponding to the same positive node pair and the corresponding negative node pairs are fused to obtain the fused similarity corresponding to the positive node pair; the negative node pair corresponding to the positive node pair is a negative node pair that has overlapping nodes with the positive node pair; based on the difference between the semantic similarity corresponding to the same positive node pair and the fused similarity, the node sub-loss corresponding to the positive node pair is obtained; based on the node sub-loss corresponding to each positive node pair, the node loss is obtained.

[0180] Feature similarity is used to characterize the degree of similarity between different initial semantic features. The greater the feature similarity, the greater the similarity between the different initial semantic features. Various similarity calculation algorithms can be used to calculate feature similarity between features. For example, cosine distance or Euclidean distance, representing the distance between features, can be used as feature similarity; the ratio of the intersection and union elements of the features can be calculated as feature similarity; and so on.

[0181] Specifically, the computer device calculates the semantic similarity corresponding to the semantic feature pair based on the feature similarity between the initial semantic features in the same semantic feature pair, and the semantic similarity and feature similarity are positively correlated. The computer device fuses the semantic similarities corresponding to the positive node pair and each negative node pair corresponding to the positive node pair to obtain the fused similarity corresponding to the positive node pair. For example, the fused similarity is obtained by adding up the individual semantic similarities; or the fused similarity is obtained by weighted summing up the individual semantic similarities. The computer device obtains the node sub-loss corresponding to the positive node pair based on the difference between the semantic similarity and the fused similarity corresponding to the same positive node pair. For example, the ratio between the semantic similarity and the fused similarity is used as the node sub-loss; or a certain multiple of the ratio between the semantic similarity and the fused similarity is used as the node sub-loss. The computer device summarizes the node sub-losses corresponding to each positive node pair to ultimately obtain the node loss.

[0182] The training sample pairs include a large number of positive and negative node pairs. There is a correspondence between the positive and negative node pairs, with negative node pairs that have overlapping nodes corresponding to positive node pairs being the corresponding negative node pairs. The network training goal is to ensure that, for a given search term, the semantic features output by the network are as similar as possible to the semantic features output by the network for the positive media data corresponding to that search term, and that the semantic features output by the network for that search term are as dissimilar as possible to the semantic features output by the network for other media data. Therefore, a correspondence is established between positive and negative node pairs that have overlapping nodes.

[0183] In one embodiment, the node loss is calculated as follows:

[0184] y q,i =h q T *h i

[0185] y q,i′ =h q T *h i′

[0186] y q′，i =h q′ T *h i

[0187]

[0188]

[0189] Among them, in the media search graph, the (q, i) edge comes from the user's historical search consumption behavior and is a natural supervision signal. Multiple (q, i) edges are sampled from the media search graph as positive node pairs. For each (q, i) positive node pair, multiple q are randomly sampled from the media search graph to replace the q in the positive node pair, and multiple (q′, i) are obtained as negative node pairs. Multiple i are randomly sampled from the media search graph to replace the i in the positive node pair, and (q, i′) is obtained as a negative node pair.

[0190] h q Represents the initial semantic features corresponding to the search term node q. i Represents the initial semantic features corresponding to media data i. q T Indicates h q The transpose of y q,i It represents the feature similarity between the initial semantic features in the (q, i) positive node pair.

[0191] h q′ Represents the initial semantic features corresponding to the search term q′. q′ T Indicates h q′ The transpose of y q′,i represents the feature similarity between the initial semantic features in the (q′, i) negative node pair.

[0192] h i′ Represents the initial semantic features corresponding to media data i′. q,i′ represents the feature similarity between the initial semantic features in the (q, i′) negative node pair.

[0193] p q,i Represents the node loss corresponding to the positive node pair. exp() represents the exponential function with the natural constant e as the base. τ represents the adjustment parameter, which can be preset or a learnable parameter in the network. exp(y q，i / τ) represents the semantic similarity corresponding to the (q, i) positive node pair. exp(y q，i′ / τ) represents the semantic similarity corresponding to the negative node pair (q, i′). exp(y q′i / τ) represents the semantic similarity corresponding to the (q′, i) negative node pair.

[0194] l sl Represents node loss.

[0195] In the above embodiment, the node loss calculated in this way can guide the network to learn to distinguish between positive node pairs and negative node pairs with corresponding relationships, so that the network outputs similar semantic features for semantically similar search terms and media data, and outputs dissimilar semantic features for semantically dissimilar search terms and media data.

[0196] In one embodiment, the perspective loss is obtained based on the difference between the sub-image feature pairs corresponding to the positive sampling sub-image pair and the negative sampling sub-image pair, including:

[0197] Based on the feature similarity between sub-image features in the same sub-image feature pair, the perspective similarity corresponding to the sub-image feature pair is obtained; the perspective similarities corresponding to the same positive sampling sub-image pair and the corresponding negative sampling sub-image pairs are fused to obtain the fused similarity corresponding to the positive sampling sub-image pair; based on the difference between the perspective similarity corresponding to the same positive sampling sub-image pair and the fused similarity, the perspective sub-loss corresponding to the positive sampling sub-image pair is obtained; based on the perspective sub-loss corresponding to each positive sampling sub-image pair, the perspective loss is obtained.

[0198] Specifically, the computer device calculates the perspective similarity corresponding to the sub-image feature pair based on the feature similarity between the sub-image features in the same sub-image feature pair, where the perspective similarity and feature similarity are positively correlated. The computer device fuses the perspective similarities corresponding to the positive sampling sub-image pair and each negative sampling sub-image pair corresponding to the positive sampling sub-image pair to obtain the fused similarity corresponding to the positive sampling sub-image pair. For example, the perspective similarities are summed to obtain the fused similarity; or the perspective similarities are weighted to obtain the fused similarity. The computer device obtains the perspective sub-loss corresponding to the positive sampling sub-image pair based on the difference between the perspective similarity and the fused similarity corresponding to the same positive sampling sub-image pair. For example, the ratio between the perspective similarity and the fused similarity is used as the perspective sub-loss; or a certain multiple of the ratio between the perspective similarity and the fused similarity is used as the perspective sub-loss. The computer device summarizes the perspective sub-losses corresponding to each positive sampling sub-image pair to ultimately obtain the perspective loss.

[0199] The second training sample pair includes a large number of positive sampling subgraph pairs and negative sampling subgraph pairs. The training goal of the network is to distinguish the subgraph features output by any different nodes. Therefore, for a certain positive sampling subgraph pair, a negative node pair can be randomly obtained as its corresponding negative sampling subgraph pair. For example, Figure 7 For the central node q1, the sampling subgraphs from different perspectives form a positive example (i.e., a positive sampling subgraph pair). For the central node q2, the sampling subgraphs from different perspectives form a positive example. The sampling subgraphs from any perspective of the central node q1 and the sampling subgraphs from any perspective of the central node q2 form a negative example (a negative sampling subgraph pair).

[0200] In one embodiment, the viewing angle loss is calculated as follows:

[0201]

[0202]

[0203] Among them, B represents the node set corresponding to a training batch, MP is the perspective set (i.e., the meta-path set). bi and bj represent two different perspectives of node b, h bi represents the subgraph feature of node b under view i, h bj Represents the subgraph features of node b under view j. ||h bi || means h bi The length of ||h bj || means h bj The length of Z bi，bj Represents the feature similarity corresponding to the positive sampling subgraph pair. b′j represents the subgraph feature of node b′ under view j. Z bi，b′j It represents the feature similarity corresponding to the negative sampling sub-image pair, which corresponds to the same perspective as the positive sampling sub-image pair.

[0204] l cl Represents the perspective loss. exp() represents the exponential function with the natural constant e as the base. τ represents the adjustment parameter, which can be preset or a learnable parameter in the network. exp(Z bi,bj / τ) represents the perspective similarity corresponding to the positive sampling sub-image pair. exp(Z bi,b′j / τ) represents the perspective similarity corresponding to the negative sampling sub-image pair.

[0205] It can be understood that the exp(Z bi，b′j / τ) can also be replaced by exp(Z bi，b′k / τ). b′k represents the subgraph feature of node b′ at view k. exp(Z bi，b′k / τ) represents the perspective similarity corresponding to the negative sampling sub-image pair, and the negative sampling sub-image pair and the positive sampling sub-image pair correspond to different perspectives.

[0206] In the above embodiment, the perspective loss calculated in this way can guide the network to learn to distinguish between positive sampling subgraph pairs and negative sampling subgraph pairs, so that the network outputs dissimilar subgraph features for sampling subgraphs belonging to different nodes.

[0207] In one embodiment, Figure 8 As shown, a search data processing method is provided, which is illustrated by applying the method to a computer device. The computer device can be a terminal or a server. The method can be executed by the terminal or the server itself, or it can be implemented through interaction between the terminal and the server. Figure 8 , the search data processing method includes the following steps:

[0208] Step S802 , obtaining a sampling subgraph corresponding to a target node; the target node includes at least one of a search term node and a media node, and the sampling subgraph corresponding to the target node is sampled from a media search graph.

[0209] The target node refers to the node whose semantic features are to be extracted. The target node can be a search term node, for example, the search term node corresponding to the search term for which the search result is to be determined. The target node can also be a media node, for example, the target node can be the media node corresponding to candidate media data, which is media data used to determine whether it is a search result for the search term.

[0210] By sampling from the media search graph based on the meta-path, a sampling subgraph corresponding to the target node can be obtained. If the target node exists in the media search graph, the corresponding sampling subgraph can be directly sampled from the media search graph based on the meta-path. If the target node does not exist in the media search graph, the node most similar to the target node can be obtained from the media search graph as a reference node, and the sampling subgraph corresponding to the reference node obtained by sampling from the media search graph based on the meta-path can be used as the sampling subgraph corresponding to the target node. The node most similar to the target node can be obtained based on the feature similarity between node features.

[0211] It can be understood that the process of constructing the media search graph can refer to the contents of the aforementioned related embodiments and will not be repeated here.

[0212] In step S804, the sampling subgraph corresponding to the target node is input into the target graph neural network to obtain the target semantic features corresponding to the target node; the target semantic features are used for data search.

[0213] The target graph neural network refers to a trained graph neural network. It is understood that the training and use processes of the graph neural network can refer to the contents of the aforementioned embodiments and will not be repeated here.

[0214] Specifically, the computer device obtains the sampling subgraph corresponding to the target node locally or from other devices, inputs the sampling subgraph corresponding to the target node into the target graph neural network, and the target graph neural network extracts features from the sampling subgraph and outputs the target semantic features corresponding to the target node.

[0215] The target semantic features corresponding to the search term node or media node can be applied to data search. For example, the target semantic features corresponding to the search term node can be applied to the q2q scenario, the target semantic features corresponding to the media node can be applied to the i2i scenario, and the target semantic features corresponding to the search term node and the media node can be applied to both the q2i scenario and the i2q scenario.

[0216] In the above-mentioned search data processing method, the training sample pairs of the graph neural network include positive node pairs and negative node pairs. The positive node pairs include interconnected search term nodes and media nodes in the media search graph, and the negative node pairs include randomly combined search term nodes and media nodes in the media search graph. The media search graph includes not only search term nodes and media nodes, but also associated nodes. The media search graph contains rich information. Based on the meta-path corresponding to the corresponding node, the sampled subgraph corresponding to the corresponding node is sampled from the media search graph. The sampled subgraph also contains rich information, which helps to improve the accuracy of subsequent feature extraction. When training the graph neural network, the positive node pairs include associated search term nodes and media nodes, and the negative node pairs include unrelated search term nodes and media nodes. Based on the differences between the semantic feature pairs corresponding to the positive node pairs and the negative node pairs, the graph neural network is trained. This can improve the accuracy of the graph neural network in feature extraction, so that the graph neural network outputs similar semantic features for associated search term nodes and media nodes, and outputs dissimilar semantic features for unrelated search term nodes and media nodes, thereby effectively improving the accuracy of subsequent searches based on semantic features. The sampling subgraph corresponding to the node is input into the trained target graph neural network for feature extraction, and the target semantic features corresponding to the node can be obtained. The target semantic features are fused with the node information corresponding to each node in the sampling subgraph, have high accuracy, and can also effectively improve the accuracy of subsequent searches.

[0217] In one embodiment, the search data processing method further includes:

[0218] Obtain the target semantic feature corresponding to the current search data of the current search object as the current search feature, and obtain the target semantic feature corresponding to the candidate search data as the candidate search feature; the current search data is the current search term or current media data, and the candidate search data is the candidate search term or candidate media data; based on the feature similarity between the current search feature and the candidate search feature, determine the target search data from multiple candidate search data as the search result corresponding to the current search data.

[0219] Among them, the current search object refers to the search user who currently triggers the data search. For example, the current search object can be a user who actively triggers a media data search by entering a search term; the current search object can be a user who passively triggers a search term search by browsing media data; and so on.

[0220] Current search data is search data determined based on the current search behavior of the current search subject. This data can be the current search term, which refers to the search term currently entered by the search subject, for example, the search term entered by the user in the search box. It can also be the current media data, which refers to the media data currently being viewed by the search subject, for example, the video currently being played by the user.

[0221] Candidate search data refers to candidate search data used to determine the search results for the current search data. The search results corresponding to the current search data are determined from multiple candidate search data. Candidate search data can be candidate search terms or candidate media data.

[0222] In the q2q scenario, the current search data is the current search term, the candidate search data is the candidate search terms, and the search term search results corresponding to the current search term are determined from multiple candidate search terms. In the q2i scenario, the current search data is the current search term, the candidate search data is the candidate media data, and the search results for media data corresponding to the current search term are determined from multiple candidate media data. In the i2i scenario, the current search data is the current media data, the candidate search data is the candidate media data, and the search results for media data corresponding to the current media data are determined from multiple candidate media data. In the i2q scenario, the current search data is the current media data, the candidate search data is the candidate search terms, and the search term search results corresponding to the current media data are determined from multiple candidate search terms.

[0223] Targeted search data refers to search results.

[0224] Specifically, in order to determine the search results corresponding to the current search data of the current search object, the computer device can obtain the target semantic features corresponding to the current search data and the target semantic features corresponding to the candidate search data based on the target graph neural network, and determine the search results of the current search data from multiple candidate search data based on the target semantic features.

[0225] The computer device obtains a target semantic feature corresponding to the current search data as the current search feature, obtains a target semantic feature corresponding to the candidate search data as the candidate search feature, and determines the target search data from the plurality of candidate search data as the search result of the current search data based on feature similarity between the current search feature and the candidate search feature. For example, from the plurality of candidate search data, at least one candidate search data with the smallest feature similarity is obtained as the target search data; and from the plurality of candidate search data, the candidate search data with a feature similarity less than a similarity threshold is obtained as the target search data.

[0226] In the above embodiment, the target semantic features output by the target graph neural network for the search terms or media data can be applied to data search for the search terms or media data to improve the accuracy of the search.

[0227] In one embodiment, based on the feature similarity between the current search feature and the candidate search feature, determining the target search data from the plurality of candidate search data as the search result corresponding to the current search data includes:

[0228] Obtain target semantic features corresponding to at least two forward media data corresponding to the current search object; fuse the target semantic features corresponding to at least two forward media data to obtain target object features corresponding to the current search object; fuse the target object features and the current search features to obtain a search fusion feature; based on the feature similarity between the search fusion feature and the candidate search features, determine the target search data from multiple candidate search data as the search result corresponding to the current search data.

[0229] The forward media data corresponding to the current search object refers to the media data that is being consumed by the current search object. For example, the forward media data is a video that has been played by the user and has a playback time greater than a certain time threshold.

[0230] The target object feature refers to the feature representation corresponding to the search object.

[0231] Specifically, in order to improve the adaptability of search results and users, user preferences can also be modeled. For example, in a personalized search scenario, user preferences can be modeled so as to return personalized search results to the user. The computer device can obtain target semantic features corresponding to at least two positive media data corresponding to the current search object; fuse the target semantic features corresponding to at least two positive media data to obtain the target object features corresponding to the current search object. For example, the target semantic features of multiple videos that the user has recently positively consumed are obtained, and each target semantic feature is averaged and normalized to obtain the user's target object features. The computer device fuses the target object features of the current search object with the target semantic features corresponding to the current search data of the current search object to obtain a search fusion feature containing semantic information of the user's preferences and the current search data, and determines the target search data from multiple candidate search data as the search result corresponding to the current search data based on the feature similarity between the search fusion feature and the target semantic features of the candidate search data.

[0232] In the above embodiment, the target object features of the search object are generated based on the target semantic features corresponding to the media data being consumed by the search object, so that the target object features include the search object's media data preference information. When performing data searches, further consideration of the target object features corresponding to the search object can match the search results with the search object, resulting in personalized search results.

[0233] In one embodiment, the search data processing method further includes:

[0234] The candidate search features corresponding to each candidate search data are input into the first processing branch of the dual-tower search model to obtain the first search features corresponding to each candidate search data; the current search feature is input into the second processing branch of the dual-tower search model to obtain the second search feature; the first search feature and the second search feature are input into the output layer of the dual-tower search model to obtain the first matching degree between each candidate search data and the current search data; based on the first matching degree, the target search data is determined from each candidate search data as the search result corresponding to the current search data.

[0235] The dual-tower search model is a two-branch machine learning model consisting of two towers, each capable of independent data processing. The input data for the dual-tower search model includes target semantic features corresponding to different search data, and the output data is the matching degree between different search data.

[0236] The dual-tower search model includes a first processing branch, a second processing branch, and an output layer. The candidate search features corresponding to the candidate search data and the current search features corresponding to the current search data are input into the dual-tower search model. The first processing branch in the dual-tower search model is used to process the candidate search features corresponding to the candidate search data, and the second processing branch is used to process the current search features corresponding to the current search data. The first processing branch and the second processing branch are used to further extract the features to enhance the expressiveness of the features. The output layer is used to calculate the matching degree between the features output by the first processing branch and the second processing branch. It can be understood that a special dual-tower search model can be set for different search scenarios, so that each search scenario has a corresponding dual-tower search model to improve the search accuracy.

[0237] The first matching degree is the matching degree output by the dual-tower search model. The matching degree is used to represent the degree of matching between different search data. The higher the matching degree, the closer the semantics between different search data are.

[0238] Specifically, a computer device can use a dual-tower search model to determine search results. The computer device inputs the candidate search features corresponding to each candidate search data into the first processing branch of the dual-tower search model. After data processing by the first processing branch, the first search features corresponding to each candidate search data are obtained. The current search feature is input into the second processing branch of the dual-tower search model. After data processing by the second processing branch, the second search feature is obtained. The computer device inputs the first search feature and the second search feature into the output layer of the dual-tower search model. After data processing by the output layer, the first matching degree between each candidate search data and the current search data is obtained. Finally, based on the first matching degree, the computer device determines the target search data from each candidate search data as the search result corresponding to the current search data. For example, a higher first matching degree indicates a higher degree of matching. From the multiple candidate search data, at least one candidate search data with the lowest first matching degree is obtained as the target search data; and from the multiple candidate search data, the candidate search data with a first matching degree less than a matching degree threshold is obtained as the target search data.

[0239] It can be understood that the candidate search features and the current search features can be input into the dual-tower search model synchronously, or the candidate search features and the current search features can be input into the dual-tower search model in a time-sharing manner.

[0240] In the above embodiment, when performing data search, the search results are determined by the dual-tower search model, which can make use of the powerful reasoning ability of the machine learning model to quickly obtain accurate search results and improve the accuracy of the search.

[0241] In one embodiment, the current search feature is input into the second processing branch of the dual-tower search model to obtain a second search feature, including:

[0242] The target object feature corresponding to the current search object and the current search feature are input into the second processing branch of the dual-tower search model to obtain a second search feature.

[0243] Specifically, to improve the suitability of search results for users, the target object features corresponding to the current search object and the current search features can be input into the second processing branch of the dual-tower search model. After data processing by the second processing branch, a second search feature is obtained. The second search feature incorporates user preference information. When subsequently determining target search data from candidate search data, user preferences can be taken into account, resulting in personalized target search data for the user.

[0244] In the above embodiment, when performing data search, the target object features of the search object are also input into the dual-tower search model to determine the search results, so that search results that not only match the semantics of the current search data, but also match the current search object can be obtained, thereby obtaining personalized and accurate search results.

[0245] In one embodiment, the data processing process of the dual-tower search model is described using a video as an example. Figure 9 The dual-tower search model includes a search term processing tower and a video processing tower. The search term processing tower is used to process the relevant information of the search term, and the video processing tower is used to process the relevant information of the video. The dual-tower search model is trained based on training data. For example, the training data includes positive node pairs and negative node pairs. The positive node pairs include the feature representations corresponding to the matching search terms and videos respectively, and the negative node pairs include the feature representations corresponding to the unmatched search terms and videos respectively. The sample pairs are input into the dual-tower search model to obtain the matching degree corresponding to the sample pairs. The model loss is calculated based on the matching degree corresponding to each sample pair, and the model parameters of the dual-tower search model are adjusted based on the model loss. It can be understood that the dual-tower search model can be trained using various training methods suitable for the dual-tower model.

[0246] The search semantic features of the search terms obtained by the method of the present application and the video semantic features obtained by the method of the present application contain richer information and can be used to improve the model performance of the traditional dual-tower search model. The search semantic features of the search terms obtained by the method of the present application are used as supplementary input data for the search term processing tower in the traditional dual-tower search model, and the video semantic features of the video obtained by the method of the present application are used as supplementary input data for the video processing tower in the traditional dual-tower search model.

[0247] After training the dual-tower search model, the dual towers and the output layer can be processed separately. The video can be pre-processed using the model's video processing tower for offline inference to produce the final feature representation and cache it. During online searches, there's no need to perform real-time inference on the video processing tower; the video's feature representation can be directly retrieved from the cache. When a video search is triggered, the search term is inferred in real time by the search term processing tower to produce the final feature representation. Finally, the final feature representation of the search term and the final feature representations of multiple candidate videos are sent to the output layer for scoring. This yields a matching score between the search term and the candidate videos. The candidate videos are then sorted from highest to lowest according to the matching score, and the final search results are determined based on the sorting results.

[0248] refer to Figure 9 In the dotted box in the personalized scenario, the search object features obtained by the method of the present application can be further used as another supplementary input data for the search term processing tower in the traditional dual-tower search model, so that the dual-tower search model has personalized capabilities.

[0249] In one embodiment, based on the first matching degree, determining target search data from each candidate search data as a search result corresponding to the current search data includes:

[0250] Based on the first matching degree, the intermediate search data is determined from each candidate search data, and the target semantic feature corresponding to the intermediate search data is used as the intermediate search feature; the current search feature and the intermediate search feature are input into the single-tower search model to obtain the second matching degree between the intermediate search data and the current search data; based on the second matching degree, the target search data is determined from each intermediate search data as the search result corresponding to the current search data.

[0251] The single-tower search model is a machine learning model that processes data in a unified manner. Its input data includes the target semantic features corresponding to different search data, and its output data includes the matching degree between different search data.

[0252] Intermediate search data refers to the search data to be further screened. The second matching degree is the matching degree output by the single tower search model.

[0253] Specifically, in order to further improve the search accuracy, after the candidate search data is preliminarily screened with the help of the dual-tower search model, the screened candidate search data can be further screened with the help of the single-tower search model to obtain the final search results.

[0254] First, the dual-tower search model performs a preliminary screening of candidate search data. Based on the first matching degree output by the dual-tower search model, the computer device determines intermediate search data from each candidate search data. For example, candidate search data with a first matching degree less than a matching degree threshold is selected as the intermediate search data. The candidate search data is sorted from highest to lowest according to the first matching degree, and a preset number of candidate search data with the highest rankings are obtained as the intermediate search data. Then, a further screening is performed using the single-tower search model. The computer device uses the target semantic features corresponding to the intermediate search data as the intermediate search features. The current search features and the intermediate search features are input into the single-tower search model. The single-tower search model fully cross-integrates the features to obtain a second matching degree between the intermediate search data and the current search data. Based on the second matching degree, the computer device determines the target search data from each intermediate search data as the search result corresponding to the current search data. For example, a higher second matching degree indicates a higher degree of matching. From the multiple intermediate search data, at least one intermediate search data with the lowest second matching degree is selected as the target search data. From the multiple intermediate search data, the intermediate search data with a second matching degree less than the matching degree threshold is selected as the target search data.

[0255] It is understandable that, when searching for data, the single-tower search model can also be used alone to filter candidate search data to obtain search results.

[0256] In the above embodiment, when performing data search, a rough search is first performed using the dual-tower search model, and then a fine search is performed using the single-tower search model, which can improve the accuracy of the search results.

[0257] In one embodiment, the current search feature and the intermediate search feature are input into a single-tower search model to obtain a second matching degree between the intermediate search data and the current search data, including:

[0258] The current search feature, the intermediate search feature, and the feature similarity between the current search feature and the intermediate search feature are input into the single tower search model to obtain a second matching degree between the intermediate search data and the current search data.

[0259] Specifically, the input data of the single-tower search model includes the target semantic features corresponding to different search data and the feature similarity between different target semantic features. The current search feature, the intermediate search feature, and the feature similarity between the current search feature and the intermediate search feature are input into the single-tower search model. The single-tower search model fully cross-integrates each input data to obtain a second matching degree between the intermediate search data and the current search data.

[0260] In the above embodiment, when searching through the single-tower search model, the feature similarity between the current search feature and the intermediate search feature is further input into the single-tower search model to provide the single-tower search model with richer input data, thereby improving the accuracy of the search results obtained by searching through the single-tower search model.

[0261] In one embodiment, the current search feature, the intermediate search feature, and the feature similarity between the current search feature and the intermediate search feature are input into a single-tower search model to obtain a second matching degree between the intermediate search data and the current search data, including:

[0262] The current search feature, the intermediate search feature, the feature similarity between the current search feature and the intermediate search feature, and the feature similarity between the target object feature corresponding to the current search object and the intermediate search feature are input into the single tower search model to obtain a second matching degree between the intermediate search data and the current search data.

[0263] Specifically, in addition to inputting the current search feature, the intermediate search feature, and the feature similarity between the current search feature and the intermediate search feature into the single-tower search model, in order to improve the compatibility of search results and users, the target object feature corresponding to the current search object and the feature similarity between the target object feature and the intermediate search feature may also be input into the single-tower search model. After data processing by the single-tower search model, a second matching degree is obtained. The target object feature corresponding to the current search object and the feature similarity between the target object feature and the intermediate search feature are input into the single-tower search model. When outputting the second matching degree, the single-tower search model further considers user preference information, so that subsequent search results determined based on the second matching degree are target search data personalized for the user.

[0264] In the above embodiment, when searching through the single-tower search model, the feature similarity between the target object features corresponding to the current search object and the intermediate search features is further input into the single-tower search model to provide the single-tower search model with richer input data, so that the search results obtained through the single-tower search model also match the search object, thereby realizing personalized search.

[0265] In one embodiment, the data processing process of the single tower search model is explained using a video as an example. Figure 10 The single-tower search model includes a feature cross layer and an output layer connected in sequence. The feature cross layer is used to cross-fuse various features, and the output layer is used to calculate the matching degree between the search term and the video based on the data output by the feature cross layer. The single-tower search model is trained based on the training data. It can be understood that the single-tower search model can be trained using various training methods suitable for the single-tower model. The search semantic features of the search term obtained by the method of the present application and the video semantic features obtained by the method of the present application contain richer information, which can be used to improve the model performance of the traditional single-tower search model.

[0266] The search semantic features of the search words obtained by the method of the present application and the video semantic features obtained by the method of the present application are used as supplementary input data of the traditional single-tower search model. In addition, a feature similarity can be calculated for the search semantic features of the search words obtained by the method of the present application and the video semantic features obtained by the method of the present application. In order to enhance the generalization ability of the feature similarity, the feature similarity is discretized and the processed feature similarity is used as another supplementary input data of the traditional single-tower search model. In addition, reference Figure 10 In the dotted box in the personalized scenario, the search object features obtained by the method of the present application can be further used as another supplementary input data of the traditional single-tower search model, so that the single-tower search model has personalized capabilities.

[0267] When applying the single-tower search model, the relevant information of the search term and candidate videos is processed through real-time inference at the feature cross-layer in the model to produce the final feature representation. Finally, the final feature representation is sent to the output layer for scoring to obtain the matching score between the search term and the candidate video. The candidate videos are sorted from large to small according to the matching score, and the final search result is determined based on the sorting result.

[0268] When a video search is triggered, the candidate videos are first roughly sorted using the dual-tower search model to obtain preliminary search results, and then the candidate videos in the initial search results are refined and re-ranked using the single-tower search model to obtain the final search results.

[0269] In a specific embodiment, referring to Figure 11 , which is a diagram of the overall architecture of the search data processing method of this application. The search data processing method of this application is used to construct a video search graph, and the graph neural network applied to the search task is trained based on the video search graph to improve search accuracy. The overall architecture of this application method includes a data module, a preprocessing module, a graph learning module, and an application module.

[0270] The data module is used to obtain raw data. Taking the video search scenario as an example, the key elements in the video search scenario are obtained to construct nodes in the video search graph. For example, the key elements include user u, search term (i.e. query) q, video v, video author a, search term or video entity e, search term or video field classification c, video tag t, etc. Obtain user search consumption behavior logs to construct edges in the video search graph. For example, under a certain query, the user clicks or plays a certain video, then establishes an edge between the query and the node corresponding to the video. Obtain entity relationship data to construct edges in the video search graph. For example, obtain the publishing relationship (the affiliation between the video and its author), obtain entity links (an entity mentioned in the query or video), and obtain labels (labels added during the video review process).

[0271] The preprocessing module is used to preprocess the raw data. The preprocessing module includes a data screening module, a data cleaning module, a composition module, and a feature processing module. For the data screening module, for user search and consumption behavior logs, the search logs in the "list page search" scenario (i.e., list search scenario) and the "immersive page search" scenario (i.e., immersive search scenario) are selected to construct a media search graph. For the data cleaning module, for search logs, logs with missing key information are cleaned out. For example, logs with missing core elements of user, query, and video are cleaned out, and logs with abnormal playback duration are cleaned out. For entity relationship data, entities with inaccurate entity recognition are removed, domain classifications that appear less frequently are removed, and the video author account system is normalized. For the composition module, the core three elements of user, query, and video in the log are used as the basic composition elements. If a user ux searches for a query qy and positively consumes a video iz, the ux, qy, and iz nodes are added to the composition, and edges are connected between them. To ensure video coverage, high-quality or highly up-to-date videos that do not appear in positive consumption behavior logs are added. Authors, entities, tags, and domain classification nodes associated with video and query nodes are added, and edges are connected to the corresponding video and query nodes in the graph. Query behavior data in search logs is mined, and adjacent queries with high text similarity and search times within a certain interval are connected in the graph. For the feature processing module, the features of nodes or edges in the graph include two types: attribute features and statistical features. For example, for nodes or edges from search behavior logs, both features need to be added, while those from other sources only need to include attribute features.

[0272] The graph learning module is used to train graph neural networks. The graph framework used to build the network can be based on computational architectures such as PlatoDeep2 and Turing. The machine learning algorithm used to build the network can be a graph algorithm such as GraphSage, DisenGCN, or GAT. The network loss calculation algorithm can use loss algorithms such as Sampled-Softmax, BPR (Bayesian Personalized Ranking, personalized ranking based on Bayesian posterior optimization), and contrastive learning. The graph neural network can output query representations (i.e., query embeddings) and video representations (i.e., video embeddings). For example, for each q and i central node (i.e., the nodes corresponding to the query and video), multiple meta-paths are designed. Each meta-path is a semantic interpretation of the associated central node, and different meta-paths correspond to different perspectives. Subgraphs surrounding the central node are sampled along multiple meta-paths, ultimately obtaining multiple sampled subgraphs corresponding to each q and i central node. The sampled subgraph corresponding to the central node is input into the graph neural network for feature aggregation to obtain the subgraph representation corresponding to the central node (i.e., subgraph feature). The subgraph representations of multiple perspectives corresponding to the central node are fused to obtain the final representation of the central node (i.e., initial semantic feature).

[0273] The network loss of a graph neural network consists of node loss and perspective loss. Gradient descent is used based on the network loss for joint training, and multiple rounds of training are performed until convergence. Similar to the generation of the final representations of the central nodes q and i during training, the trained graph neural network is used to produce and fuse multi-perspective representations of the nodes q and i. The normalized results serve as the final embedding of the query (i.e., the target semantic features of the query) and the final embedding of the video (i.e., the target semantic features of the video). For personalized application scenarios, user preferences need to be modeled. The final embeddings of the N videos that the user has recently consumed are averaged and normalized to serve as the user representation of the user.

[0274] For the search data processing method of this application, the search object behavior and domain entity association relationship related to the video search scene are fully considered, and the search object behavior supervised learning and multi-view graph comparison learning method are combined for joint pre-training to learn more general information of the video search scene.

[0275] The application module applies the trained graph neural network to various search-related tasks to improve the accuracy of search results. The graph neural network extracts feature vectors (i.e., target semantic features) corresponding to search terms and videos, and performs data recall based on these feature vectors. Furthermore, the graph neural network can also extract feature vectors corresponding to users, further incorporating user representations during data recall to achieve personalized search. The trained graph neural network can be applied to query recommendation tasks. For example, in i2q search scenarios, it can be used to retrieve relevant search terms based on the videos a user is currently consuming, thereby recommending relevant search terms to the user. The trained graph neural network can also be applied to short video recall tasks. For example, in q2i search scenarios, it can be used to retrieve relevant short videos based on the user's current search term, thereby returning short videos found based on the current search term. In i2i search scenarios, it can be used to retrieve related short videos based on the short videos a user is currently consuming, thereby recommending relevant short videos to the user. In short video recall, the short video recall model can be used, and the feature vectors extracted by the graph neural network can be used as the input features of the short video recall model to improve the accuracy of the short video recall model. The trained graph neural network can be applied to (QU, Query Understanding) tasks. For example, in the q2q search scenario, related search terms can be recalled based on the user's current search terms, thereby recommending related search terms to the user. In any search task, the candidate data can be roughly sorted, finely sorted, and re-ranked to obtain the final search results. In the rough sorting, fine sorting, and re-ranking, the ranking model can be used, and the feature vectors extracted by the graph neural network can be used as the input features of the ranking model to improve the accuracy of the ranking model.

[0276] In a specific embodiment, the search data processing method of the present application can be applied to video search scenarios. Figure 12In the q2i search scenario, after the user inputs a search term (i.e., query) to initiate a video retrieval, it will be processed by the search term understanding module (QU, Query Understanding). In the QU module, the final search results are obtained by sequentially passing through the recall, coarse sorting, fine sorting, and rearrangement modules to be presented to the user. The target semantic features extracted by the method of the present application can be applied to optimize any module in the QU module. For example, the target semantic features of the search term or video extracted by the method of the present application can be used as input features of relevant models in the recall, coarse sorting, fine sorting, and rearrangement modules to supplement the model with deep semantics and interactive information and improve its effect. In the QU module, the target semantic features of the query and the target semantic features of the candidate videos in the video library are extracted by the method of the present application. Based on the target semantic features of the query and the candidate videos, the candidate videos related to the query are recalled from the video library, and the recalled candidate videos are sequentially screened by the coarse sorting, fine sorting, and rearrangement modules. Finally, several target videos are obtained as the search results of the query and presented to the user. For example, recall is performed based on the feature similarity between the target semantic features of the query and candidate videos, coarse ranking is performed based on the dual-tower search model, fine ranking is performed based on the single-tower search model, and re-ranking is performed based on the single-tower search model. Another example is recall based on the search-recall model, coarse ranking is performed based on the dual-tower search model, fine ranking is performed based on the single-tower search model, and re-ranking is performed based on the single-tower search model. The input data of the search-recall model includes the target semantic features of the query and candidate videos.

[0277] It is understood that for video search, in addition to being applied to q2i search scenarios, it can also be applied to various search scenarios such as q2q, i2q, and i2i. In addition to media data such as video, the search data processing method of this application can be applied to various search scenarios for other media data.

[0278] Furthermore, for the recall module, queries or videos are recalled from the recall pool. Regardless of whether the query or video is being recalled, it is generally necessary to select queries or videos within a certain range to establish an index library. The video library selection criteria consider factors such as video quality, timeliness, and the number of views / completion rate in recent days. The query library selection criteria consider factors such as the number of views / completion rate of videos under the query. After the query or video range is selected, a vector index can be established. Commonly used vector index engines include Faiss (Facebook AI Similarity Search, an open source clustering and similarity search library developed by the Facebook AI team), Annoy (APPROXIMATE NEAREST NEIGHBORS OH YEAH, approximate nearest neighbor search algorithm), etc.

[0279] Video search scenarios involve a wealth of information on entities, elements, their relationships with each other, and user interaction behaviors. The method of this application sorts out this information and effectively integrates it based on the "user-query-video" ternary relationship. It also combines supervised learning of search object behavior with multi-view graph comparison learning methods for joint pre-training, thereby learning richer and more general query and short video representations, supplementing the deficiencies in relevant signals in current search scenario algorithms.

[0280] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0281] Based on the same inventive concept, embodiments of the present application further provide a search data processing device for implementing the aforementioned search data processing method. The implementation solution provided by this device is similar to the implementation solution described in the aforementioned method. Therefore, the specific limitations in one or more search data processing device embodiments provided below can be found in the above-described limitations on the search data processing method and will not be further elaborated here.

[0282] In one embodiment, Figure 13 As shown, a search data processing device 1300 is provided, comprising: a media search graph acquisition module 1302, a training sample pair acquisition module 1304, a media search graph sampling module 1306, an initial semantic feature determination module 1308, and a network training module 1310, wherein:

[0283] The media search graph acquisition module 1302 is used to obtain a media search graph; the media search graph includes a search term node corresponding to the search term, a media node corresponding to the media data, and an association node corresponding to the associated data, and the associated data includes data associated with at least one of the search term and the media data.

[0284] The training sample pair acquisition module 1304 is used to obtain multiple first training sample pairs from the media search graph; the multiple first training sample pairs include positive node pairs and negative node pairs, the positive node pairs include search term nodes and media nodes that are interconnected in the media search graph, and the negative node pairs include search term nodes and media nodes that are randomly combined in the media search graph.

[0285] The media search graph sampling module 1306 is used to sample the media search graph based on the meta-paths corresponding to the search terms and media data, respectively, to obtain sampling sub-graphs corresponding to the search term nodes and media nodes in the first training sample pair; the meta-path is a sampling path starting from the search term node or media node in the media search graph.

[0286] The initial semantic feature determination module 1308 is used to input the sampling subgraphs corresponding to the search term nodes and media nodes in the first training sample pair into the initial graph neural network, obtain the initial semantic features of the search term nodes and media nodes in the first training sample pair, and form the semantic feature pairs corresponding to the first training sample pair.

[0287] The network training module 1310 is used to train the initial graph neural network based on the differences between the semantic feature pairs corresponding to the positive node pairs and the negative node pairs to obtain a target graph neural network; the target graph neural network is used to determine the target semantic features corresponding to the search term nodes or media nodes.

[0288] In one embodiment, the media search graph acquisition module 1302 is further configured to:

[0289] Obtain historical search information corresponding to multiple search objects; the historical search information includes historical search terms and positive media data corresponding to the historical search terms, and the positive media data is the media data of the positive feedback of the search objects; generate object nodes corresponding to each search object, search term nodes corresponding to each historical search term, and media nodes corresponding to each positive media data, establish a connection relationship between the object node and the corresponding search term node, establish a connection relationship between the object node and the corresponding media node, and establish a connection relationship between the search term node and the corresponding media node; obtain at least one associated data corresponding to the historical search term and the positive media data; at least one associated data includes an associated entity, an associated tag, an associated category or an associated publishing object; generate an associated node corresponding to each associated data, establish a connection relationship between the search term node and the corresponding associated node, and establish a connection relationship between the media node and the corresponding associated node; obtain node features corresponding to each node and connection features corresponding to each connection relationship; generate a media search graph based on each node, the node features corresponding to the nodes, the connection relationship between the nodes, and the connection features corresponding to the connection relationship.

[0290] In one embodiment, the media search graph acquisition module 1302 is further configured to:

[0291] Obtain supplementary media data and generate a media node corresponding to the supplementary media data; the supplementary media data includes at least one of first media data and second media data, the first media data refers to media data whose media quality is greater than a quality threshold, and the second media data refers to media data whose time interval between a published time and a current time is less than a time interval threshold; obtain at least one associated data corresponding to the supplementary media data.

[0292] In one embodiment, the media search graph acquisition module 1302 is further configured to:

[0293] Obtain a rewritten search term corresponding to a historical search term; when the search time interval between the historical search term and the corresponding rewritten search term is less than a time interval threshold, and the similarity between the historical search term and the corresponding rewritten search term is greater than a similarity threshold, generate a search term node corresponding to the rewritten search term; establish a connection relationship between the search term node corresponding to the historical search term and the search term node corresponding to the rewritten search term.

[0294] In one embodiment, the media search graph sampling module 1306 is further configured to:

[0295] Determine a current search meta-path from at least one meta-path corresponding to a search term; the current search meta-path is a path formed by sequentially connecting type flags corresponding to the search type, the first type, and the second type, respectively; take the current search term node as the search center node, sample at least two nodes of the first type directly connected to the search center node from the media search graph as first-order neighbor nodes corresponding to the search center node, sample at least two nodes of the second type directly connected to the first-order neighbor node from the media search graph as second-order neighbor nodes corresponding to the search center node, and obtain a sampling subgraph corresponding to the search center node under the current search meta-path based on the search center node and the corresponding first and second neighbor nodes;

[0296] Determine the current media meta-path from at least one meta-path corresponding to the media data; the current media meta-path is a path formed by sequentially connecting type identifiers corresponding to the media type, the third type, and the fourth type; take the current media node as the media center node, sample at least two nodes of the third type directly connected to the media center node from the search media graph as first-order neighbor nodes corresponding to the media center node, sample at least two nodes of the fourth type directly connected to the first-order neighbor node from the search media graph as second-order neighbor nodes corresponding to the media center node, and based on the media center node and the corresponding first neighbor node and second neighbor node, obtain a sampling sub-graph corresponding to the media center node under the current media meta-path.

[0297] In one embodiment, the current node is a search term node or a media node. The initial semantic feature determination module 1308 is further configured to:

[0298] The sampling subgraphs of at least two perspectives corresponding to the current node are input into the initial graph neural network to obtain the subgraph features of each sampling subgraph corresponding to the current node; the current node is the search term node or media node in the first training sample pair, and different meta-paths correspond to different semantic perspectives; the subgraph features corresponding to the current node are fused to obtain the initial semantic features corresponding to the current node.

[0299] In one embodiment, the sampling subgraph corresponding to the node includes the node and the first-order neighbor nodes and second-order neighbor nodes corresponding to the node. The initial semantic feature determination module 1308 is also used to:

[0300] Through the initial graph neural network, the node features corresponding to the second-order neighbor nodes in the current sampling subgraph corresponding to the current node, as well as the connection features between the second-order neighbor nodes and the first-order neighbor nodes are aggregated to the first-order neighbor nodes to obtain the second-order aggregate features corresponding to the first-order neighbor nodes; through the initial graph neural network, the node features corresponding to the first-order neighbor nodes, the second-order aggregate features, and the connection features between the first-order neighbor nodes and the current node are aggregated to the current node to obtain the first-order aggregate features corresponding to the current node; through the initial graph neural network, based on the node features and first-order aggregate features corresponding to the current node, the subgraph features of the current sampling subgraph corresponding to the current node are obtained.

[0301] In one embodiment, the network training module 1310 is further configured to:

[0302] Based on the difference between the semantic feature pairs corresponding to the positive node pair and the negative node pair, a node loss is obtained; a second training sample pair is obtained; the second training sample pair includes a positive sampling subgraph pair and a negative sampling subgraph pair, the positive sampling subgraph pair includes sampling subgraphs of different semantic perspectives corresponding to the same node, and the negative sampling subgraph pair includes sampling subgraphs corresponding to different nodes; each sampling subgraph in the second training sample pair is input into the initial graph neural network to obtain the subgraph features of each sampling subgraph in the second training sample pair, which constitute the subgraph feature pairs corresponding to the second training sample pair; based on the difference between the subgraph feature pairs corresponding to the positive sampling subgraph pair and the negative sampling subgraph pair, a perspective loss is obtained; based on the node loss and the perspective loss, the initial graph neural network is trained to obtain the target graph neural network.

[0303] In one embodiment, the network training module 1310 is further configured to:

[0304] Based on the feature similarity between the initial semantic features in the same semantic feature pair, the semantic similarity corresponding to the semantic feature pair is obtained; the semantic similarities corresponding to the same positive node pair and the corresponding negative node pairs are fused to obtain the fused similarity corresponding to the positive node pair; the negative node pair corresponding to the positive node pair is a negative node pair that has overlapping nodes with the positive node pair; based on the difference between the semantic similarity corresponding to the same positive node pair and the fused similarity, the node sub-loss corresponding to the positive node pair is obtained; based on the node sub-loss corresponding to each positive node pair, the node loss is obtained.

[0305] In one embodiment, the network training module 1310 is further configured to:

[0306] Based on the feature similarity between sub-image features in the same sub-image feature pair, the perspective similarity corresponding to the sub-image feature pair is obtained; the perspective similarities corresponding to the same positive sampling sub-image pair and the corresponding negative sampling sub-image pairs are fused to obtain the fused similarity corresponding to the positive sampling sub-image pair; based on the difference between the perspective similarity corresponding to the same positive sampling sub-image pair and the fused similarity, the perspective sub-loss corresponding to the positive sampling sub-image pair is obtained; based on the perspective sub-loss corresponding to each positive sampling sub-image pair, the perspective loss is obtained.

[0307] In the above-mentioned search data processing device, the media search graph includes not only search term nodes and media nodes, but also associated nodes. The media search graph contains rich information. Then, based on the meta-path corresponding to the corresponding node, a sampled subgraph corresponding to the corresponding node is sampled from the media search graph. The sampled subgraph also contains rich information, which helps to improve the accuracy of subsequent feature extraction. The sampled subgraph corresponding to the node is input into the trained target graph neural network for feature extraction, and the target semantic feature corresponding to the node can be obtained. The target semantic feature is integrated with the node information corresponding to each node in the sampled subgraph, has high accuracy, and can also effectively improve the accuracy of subsequent searches. In addition, when training the graph neural network, positive node pairs include associated search term nodes and media nodes, and negative node pairs include unrelated search term nodes and media nodes. Based on the differences between the semantic feature pairs corresponding to the positive node pairs and the negative node pairs, the graph neural network is trained, which can improve the accuracy of the graph neural network in feature extraction, so that the graph neural network outputs similar semantic features for associated search term nodes and media nodes, and outputs dissimilar semantic features for unrelated search term nodes and media nodes, thereby effectively improving the accuracy of subsequent searches based on semantic features.

[0308] In one embodiment, Figure 14 As shown, a search data processing device 1400 is provided, comprising: an image acquisition module 1402 and a feature determination module 1404, wherein:

[0309] The image acquisition module 1402 is used to acquire a sampling subgraph corresponding to a target node; the target node includes at least one of a search term node and a media node, and the sampling subgraph corresponding to the target node is sampled from a media search graph.

[0310] The feature determination module 1404 is used to input the sampling subgraph corresponding to the target node into the target graph neural network to obtain the target semantic feature corresponding to the target node; the target semantic feature is used for data search.

[0311] Among them, the training process of the target graph neural network can refer to the contents of the aforementioned embodiments and will not be repeated here.

[0312] In one embodiment, Figure 15 As shown, the search data processing device 1400 further includes a data search module 1406, which is further configured to:

[0313] Obtain the target semantic feature corresponding to the current search data of the current search object as the current search feature, and obtain the target semantic feature corresponding to the candidate search data as the candidate search feature; the current search data is the current search term or current media data, and the candidate search data is the candidate search term or candidate media data; based on the feature similarity between the current search feature and the candidate search feature, determine the target search data from multiple candidate search data as the search result corresponding to the current search data.

[0314] In one embodiment, the data search module 1406 is further configured to:

[0315] Obtain target semantic features corresponding to at least two forward media data corresponding to the current search object; fuse the target semantic features corresponding to at least two forward media data to obtain target object features corresponding to the current search object; fuse the target object features and the current search features to obtain a search fusion feature; based on the feature similarity between the search fusion feature and the candidate search features, determine the target search data from multiple candidate search data as the search result corresponding to the current search data.

[0316] In one embodiment, the data search module 1406 is further configured to:

[0317] The candidate search features corresponding to each candidate search data are input into the first processing branch of the dual-tower search model to obtain the first search features corresponding to each candidate search data; the current search feature is input into the second processing branch of the dual-tower search model to obtain the second search feature; the first search feature and the second search feature are input into the output layer of the dual-tower search model to obtain the first matching degree between each candidate search data and the current search data; based on the first matching degree, the target search data is determined from each candidate search data as the search result corresponding to the current search data.

[0318] In one embodiment, the data search module 1406 is further configured to:

[0319] The target object feature corresponding to the current search object and the current search feature are input into the second processing branch of the dual-tower search model to obtain a second search feature.

[0320] In one embodiment, the data search module 1406 is further configured to:

[0321] Based on the first matching degree, the intermediate search data is determined from each candidate search data, and the target semantic feature corresponding to the intermediate search data is used as the intermediate search feature; the current search feature and the intermediate search feature are input into the single-tower search model to obtain the second matching degree between the intermediate search data and the current search data; based on the second matching degree, the target search data is determined from each intermediate search data as the search result corresponding to the current search data.

[0322] In one embodiment, the data search module 1406 is further configured to:

[0323] The current search feature, the intermediate search feature, and the feature similarity between the current search feature and the intermediate search feature are input into the single tower search model to obtain a second matching degree between the intermediate search data and the current search data.

[0324] In one embodiment, the data search module 1406 is further configured to:

[0325] The current search feature, the intermediate search feature, the feature similarity between the current search feature and the intermediate search feature, and the feature similarity between the target object feature corresponding to the current search object and the intermediate search feature are input into the single tower search model to obtain a second matching degree between the intermediate search data and the current search data.

[0326] In the search data processing device described above, the training sample pairs of the graph neural network include positive node pairs and negative node pairs. The positive node pairs include interconnected search term nodes and media nodes in the media search graph, and the negative node pairs include randomly combined search term nodes and media nodes in the media search graph. The media search graph includes not only search term nodes and media nodes, but also associated nodes. The media search graph contains rich information. Based on the meta-path corresponding to the corresponding node, a sampled subgraph corresponding to the corresponding node is sampled from the media search graph. The sampled subgraph also contains rich information, which helps improve the accuracy of subsequent feature extraction. When training the graph neural network, the positive node pairs include associated search term nodes and media nodes, and the negative node pairs include unrelated search term nodes and media nodes. Training the graph neural network based on the differences between the semantic feature pairs corresponding to the positive node pairs and the negative node pairs can improve the accuracy of the graph neural network in feature extraction, so that the graph neural network outputs similar semantic features for associated search term nodes and media nodes, and outputs dissimilar semantic features for unrelated search term nodes and media nodes, thereby effectively improving the accuracy of subsequent searches based on semantic features. The sampling subgraph corresponding to the node is input into the trained target graph neural network for feature extraction, and the target semantic features corresponding to the node can be obtained. The target semantic features are fused with the node information corresponding to each node in the sampling subgraph, have high accuracy, and can also effectively improve the accuracy of subsequent searches.

[0327] Each module in the search data processing device described above may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0328] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 16As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store data such as media search graphs and graph neural networks. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a search data processing method is implemented.

[0329] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 17 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a search data processing method is implemented. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse, etc.

[0330] Those skilled in the art will understand that Figure 16 、 17The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0331] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0332] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above-mentioned method embodiments when executed by a processor.

[0333] In one embodiment, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0334] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0335] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0336] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0337] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A search data processing method, characterized in that: The method comprises: Obtaining a media search graph; the media search graph includes a search term node corresponding to a search term, a media node corresponding to media data, and an association node corresponding to association data, wherein the association data includes data associated with at least one of the search term and the media data; Acquire a plurality of first training sample pairs from the media search graph; the plurality of first training sample pairs include positive node pairs and negative node pairs, the positive node pairs include search term nodes and media nodes connected to each other in the media search graph, and the negative node pairs include search term nodes and media nodes randomly combined in the media search graph; Determining a current search meta-path from at least one meta-path corresponding to the search term; the current search meta-path is a path formed by sequentially connecting type identifiers corresponding to the search type, the first type, and the second type; the meta-path is a sampling path starting from a search term node or a media node in the media search graph; Taking the current search term node as the search center node, sampling at least two nodes of the first type that are directly connected to the search center node from the media search graph as the first-order neighbor nodes corresponding to the search center node, sampling at least two nodes of the second type that are directly connected to the first-order neighbor node from the media search graph as the second-order neighbor nodes corresponding to the search center node, and obtaining a sampling subgraph corresponding to the search center node under the current search meta-path based on the search center node and the corresponding first and second neighbor nodes; Determining a current media meta-path from at least one meta-path corresponding to the media data; the current media meta-path is a path formed by sequentially connecting type identifiers corresponding to the media type, the third type, and the fourth type; Taking the current media node as the media center node, sampling at least two nodes of the third type that are directly connected to the media center node from the media search graph as first-order neighbor nodes corresponding to the media center node, sampling at least two nodes of the fourth type that are directly connected to the first-order neighbor node from the media search graph as second-order neighbor nodes corresponding to the media center node, and obtaining a sampling subgraph corresponding to the media center node under the current media element path based on the media center node and the corresponding first and second neighbor nodes; Inputting the sampling subgraphs corresponding to the search term node and the media node in the first training sample pair into the initial graph neural network, obtaining the initial semantic features of the search term node and the media node in the first training sample pair, and forming the semantic feature pairs corresponding to the first training sample pair; Based on the differences between the semantic feature pairs corresponding to the positive node pairs and the negative node pairs, the initial graph neural network is trained to obtain a target graph neural network; the target graph neural network is used to determine the target semantic features corresponding to the search term nodes or media nodes.

2. The method according to claim 1, characterized in that The obtaining of the media search graph includes: Acquire historical search information corresponding to multiple search objects; the historical search information includes historical search terms and positive media data corresponding to the historical search terms, the positive media data being media data positively fed back by the search objects; Generate object nodes corresponding to each search object, search term nodes corresponding to each historical search term, and media nodes corresponding to each forward media data, establish a connection relationship between the object node and the corresponding search term node, establish a connection relationship between the object node and the corresponding media node, and establish a connection relationship between the search term node and the corresponding media node; Obtaining at least one associated data corresponding to each of the historical search term and the forward media data; the at least one associated data includes an associated entity, an associated tag, an associated category, or an associated publishing object; Generate associated nodes corresponding to each associated data, establish a connection relationship between the search word node and the corresponding associated node, and establish a connection relationship between the media node and the corresponding associated node; Obtain the node features corresponding to each node and the connection features corresponding to each connection relationship; A media search graph is generated based on the nodes, the node features corresponding to the nodes, the connection relationships between the nodes, and the connection features corresponding to the connection relationships.

3. The method according to claim 2, characterized in that Before generating the associated nodes corresponding to the respective associated data, the method further includes: Acquire supplementary media data and generate a media node corresponding to the supplementary media data; the supplementary media data includes at least one of first media data and second media data, the first media data being media data having a media quality greater than a quality threshold, and the second media data being media data having a time interval between a published time and a current time less than a time interval threshold; At least one associated data corresponding to the supplementary media data is obtained.

4. The method according to claim 2, characterized in that The method further comprises: Obtaining a rewritten search term corresponding to the historical search term; When the search time interval between the historical search term and the corresponding rewritten search term is less than a time interval threshold, and the similarity between the historical search term and the corresponding rewritten search term is greater than a similarity threshold, generating a search term node corresponding to the rewritten search term; A connection relationship is established between the search term node corresponding to the historical search term and the search term node corresponding to the rewritten search term.

5. The method according to claim 1, wherein The step of inputting the sampling subgraphs corresponding to the search term node and the media node in the first training sample pair into the initial graph neural network to obtain initial semantic features of the search term node and the media node in the first training sample pair includes: Inputting sampling subgraphs of at least two semantic perspectives corresponding to a current node into an initial graph neural network to obtain subgraph features of each sampling subgraph corresponding to the current node; the current node is a search term node or a media node in the first training sample pair, and different meta-paths correspond to different semantic perspectives; The subgraph features corresponding to the current node are fused to obtain the initial semantic feature corresponding to the current node.

6. The method according to claim 5, characterized in that The sampling subgraph corresponding to the node includes the node and the first-order neighbor nodes and second-order neighbor nodes corresponding to the node. The sampling subgraphs of at least two semantic perspectives corresponding to the current node are input into the initial graph neural network to obtain the subgraph features of each sampling subgraph corresponding to the current node, including: Aggregating, through the initial graph neural network, node features corresponding to second-order neighbor nodes in the current sampling subgraph corresponding to the current node, and connection features between the second-order neighbor nodes and the first-order neighbor nodes to the first-order neighbor nodes, to obtain second-order aggregated features corresponding to the first-order neighbor nodes; Aggregating the node features, second-order aggregate features, and connection features between the first-order neighbor nodes and the current node to the current node through the initial graph neural network to obtain the first-order aggregate features corresponding to the current node; Through the initial graph neural network, based on the node features and first-order aggregation features corresponding to the current node, the subgraph features of the current sampling subgraph corresponding to the current node are obtained.

7. The method according to any one of claims 1 to 6, characterized in that The training of the initial graph neural network based on the difference between the semantic feature pairs corresponding to the positive node pairs and the negative node pairs to obtain a target graph neural network includes: Obtaining a node loss based on a difference between semantic feature pairs corresponding to the positive node pair and the negative node pair; Acquire multiple second training sample pairs; the multiple second training sample pairs include positive sampling subgraph pairs and negative sampling subgraph pairs, the positive sampling subgraph pairs include sampling subgraphs of different semantic perspectives corresponding to the same node, and the negative sampling subgraph pairs include sampling subgraphs corresponding to different nodes of the same type; Inputting each sampling subgraph in the second training sample pair into the initial graph neural network to obtain the subgraph features of each sampling subgraph in the second training sample pair, forming a subgraph feature pair corresponding to the second training sample pair; Obtaining a perspective loss based on differences between sub-image feature pairs corresponding to the positive sampling sub-image pair and the negative sampling sub-image pair; Based on the node loss and the perspective loss, the initial graph neural network is trained to obtain a target graph neural network.

8. The method according to claim 7, characterized in that Obtaining the node loss based on the difference between the semantic feature pairs corresponding to the positive node pair and the negative node pair, respectively, includes: Based on the feature similarity between the initial semantic features in the same semantic feature pair, obtaining the semantic similarity corresponding to the semantic feature pair; The semantic similarities corresponding to the same positive node pair and the corresponding negative node pairs are fused to obtain the fused similarity corresponding to the positive node pair; the negative node pair corresponding to the positive node pair is a negative node pair that has overlapping nodes with the positive node pair; Based on the difference between the semantic similarity and the fusion similarity corresponding to the same positive node pair, the node sub-loss corresponding to the positive node pair is obtained; Based on the node sub-loss corresponding to each positive node pair, the node loss is obtained.

9. The method according to claim 7, characterized in that The obtaining of the perspective loss based on the difference between the sub-image feature pairs corresponding to the positive sampling sub-image pair and the negative sampling sub-image pair includes: Based on the feature similarity between the sub-image features in the same sub-image feature pair, obtaining the viewpoint similarity corresponding to the sub-image feature pair; Fusing the perspective similarities corresponding to the same positive sampling sub-image pair and the corresponding negative sampling sub-image pairs to obtain the fused similarity corresponding to the positive sampling sub-image pair; Obtaining a view sub-loss corresponding to the same positive sampling sub-image pair based on a difference between the view similarity and the fusion similarity corresponding to the positive sampling sub-image pair; Based on the view sub-loss corresponding to each positive sampling sub-image pair, the view loss is obtained.

10. A search data processing method, characterized in that: The method comprises: Obtaining a sampling subgraph corresponding to a target node; the target node includes at least one of a search term node and a media node, and the sampling subgraph corresponding to the target node is sampled from a media search graph; Inputting the sampling subgraph corresponding to the target node into the target graph neural network to obtain the target semantic feature corresponding to the target node; the target semantic feature is used for data search; The training process of the target graph neural network includes the following steps: Obtaining the media search graph; the media search graph includes a search term node corresponding to the search term, a media node corresponding to the media data, and an association node corresponding to the association data, wherein the association data includes data associated with at least one of the search term and the media data; Acquire a plurality of first training sample pairs from the media search graph; the plurality of first training sample pairs include positive node pairs and negative node pairs, the positive node pairs include search term nodes and media nodes connected to each other in the media search graph, and the negative node pairs include search term nodes and media nodes randomly combined in the media search graph; Determining a current search meta-path from at least one meta-path corresponding to the search term; the current search meta-path is a path formed by sequentially connecting type identifiers corresponding to the search type, the first type, and the second type; the meta-path is a sampling path starting from a search term node or a media node in the media search graph; Taking the current search term node as the search center node, sampling at least two nodes of the first type that are directly connected to the search center node from the media search graph as the first-order neighbor nodes corresponding to the search center node, sampling at least two nodes of the second type that are directly connected to the first-order neighbor node from the media search graph as the second-order neighbor nodes corresponding to the search center node, and obtaining a sampling subgraph corresponding to the search center node under the current search meta-path based on the search center node and the corresponding first and second neighbor nodes; Determining a current media meta-path from at least one meta-path corresponding to the media data; the current media meta-path is a path formed by sequentially connecting type identifiers corresponding to the media type, the third type, and the fourth type; Taking the current media node as the media center node, sampling at least two nodes of the third type that are directly connected to the media center node from the media search graph as first-order neighbor nodes corresponding to the media center node, sampling at least two nodes of the fourth type that are directly connected to the first-order neighbor node from the media search graph as second-order neighbor nodes corresponding to the media center node, and obtaining a sampling subgraph corresponding to the media center node under the current media element path based on the media center node and the corresponding first and second neighbor nodes; Inputting the sampling subgraphs corresponding to the search term node and the media node in the first training sample pair into the initial graph neural network, obtaining the initial semantic features of the search term node and the media node in the first training sample pair, and forming the semantic feature pairs corresponding to the first training sample pair; Based on the differences between the semantic feature pairs corresponding to the positive node pairs and the negative node pairs, the initial graph neural network is trained to obtain a target graph neural network.

11. The method according to claim 10, characterized in that The method further comprises: Obtaining a target semantic feature corresponding to current search data of a current search object as a current search feature, and obtaining a target semantic feature corresponding to candidate search data as a candidate search feature; the current search data is a current search term or current media data, and the candidate search data is a candidate search term or candidate media data; Based on the feature similarity between the current search feature and the candidate search feature, target search data is determined from a plurality of candidate search data as the search result corresponding to the current search data.

12. The method according to claim 11, characterized in that The determining, based on the feature similarity between the current search feature and the candidate search feature, target search data from a plurality of candidate search data as a search result corresponding to the current search data includes: Obtain target semantic features corresponding to at least two pieces of forward media data corresponding to the current search object; Fusing the target semantic features corresponding to the at least two forward media data to obtain a target object feature corresponding to the current search object; Fusing the target object feature and the current search feature to obtain a search fusion feature; Based on the feature similarity between the search fusion feature and the candidate search feature, target search data is determined from a plurality of candidate search data as the search result corresponding to the current search data.

13. The method according to claim 11, characterized in that The method further comprises: Inputting candidate search features corresponding to each candidate search data into the first processing branch of the dual-tower search model to obtain first search features corresponding to each candidate search data; Inputting the current search feature into the second processing branch of the dual-tower search model to obtain a second search feature; Inputting the first search feature and the second search feature into the output layer of the dual-tower search model to obtain a first matching degree between each candidate search data and the current search data; Based on the first matching degree, target search data is determined from the candidate search data as the search result corresponding to the current search data.

14. The method according to claim 13, characterized in that The determining, based on the first matching degree, target search data from each candidate search data as a search result corresponding to the current search data includes: Based on the first matching degree, determining intermediate search data from each candidate search data, and using the target semantic feature corresponding to the intermediate search data as the intermediate search feature; Inputting the current search feature and the intermediate search feature into a single-tower search model to obtain a second matching degree between the intermediate search data and the current search data; Based on the second matching degree, target search data is determined from each intermediate search data as the search result corresponding to the current search data.

15. A search data processing device, characterized in that: The device comprises: a media search graph acquisition module, configured to acquire a media search graph; the media search graph comprising a search term node corresponding to a search term, a media node corresponding to media data, and an association node corresponding to association data, wherein the association data comprises data associated with at least one of the search term and the media data; a training sample pair acquisition module, configured to acquire a plurality of first training sample pairs from the media search graph; the plurality of first training sample pairs comprising positive node pairs and negative node pairs, the positive node pairs comprising search term nodes and media nodes interconnected in the media search graph, and the negative node pairs comprising search term nodes and media nodes randomly combined in the media search graph; A media search graph sampling module is used to determine a current search meta-path from at least one meta-path corresponding to the search term; the current search meta-path is a path formed by sequentially connecting type flags corresponding to the search type, the first type, and the second type; the meta-path is a sampling path starting from a search term node or a media node in the media search graph; taking the current search term node as a search center node, sampling at least two nodes of the first type directly connected to the search center node from the media search graph as first-order neighbor nodes corresponding to the search center node, sampling at least two nodes of the second type directly connected to the first-order neighbor node from the media search graph as second-order neighbor nodes corresponding to the search center node, and obtaining the search term node based on the search center node and the corresponding first neighbor node and second neighbor node. A sampling subgraph corresponding to the central node under the current search meta-path; determining the current media meta-path from at least one meta-path corresponding to the media data; the current media meta-path is a path formed by sequentially connecting type identifiers corresponding to the media type, the third type, and the fourth type; taking the current media node as the media center node, sampling at least two nodes of the third type directly connected to the media center node from the media search graph as first-order neighbor nodes corresponding to the media center node, sampling at least two nodes of the fourth type directly connected to the first-order neighbor node from the media search graph as second-order neighbor nodes corresponding to the media center node, and obtaining the sampling subgraph corresponding to the media center node under the current media meta-path based on the media center node and the corresponding first neighbor node and second neighbor node; an initial semantic feature determination module, configured to input the sampling subgraphs corresponding to the search term node and the media node in the first training sample pair into an initial graph neural network, obtain initial semantic features of the search term node and the media node in the first training sample pair, and form a semantic feature pair corresponding to the first training sample pair; A network training module is used to train the initial graph neural network based on the differences between the semantic feature pairs corresponding to the positive node pairs and the negative node pairs to obtain a target graph neural network; the target graph neural network is used to determine the target semantic features corresponding to the search term nodes or media nodes.

16. The search data processing device according to claim 15, characterized in that: The media search graph acquisition module is also used to obtain historical search information corresponding to multiple search objects; the historical search information includes historical search terms and positive media data corresponding to the historical search terms, and the positive media data is media data of positive feedback of the search object; generate object nodes corresponding to each search object, search term nodes corresponding to each historical search term, and media nodes corresponding to each positive media data, establish connection relationships between object nodes and corresponding search term nodes, establish connection relationships between object nodes and corresponding media nodes, and establish connection relationships between search term nodes and corresponding media nodes; obtain at least one associated data corresponding to the historical search terms and the positive media data; the at least one associated data includes associated entities, associated tags, associated categories or associated publishing objects; generate associated nodes corresponding to each associated data, establish connection relationships between search term nodes and corresponding associated nodes, and establish connection relationships between media nodes and corresponding associated nodes; obtain node features corresponding to each node and connection features corresponding to each connection relationship; generate a media search graph based on each node, node features corresponding to the nodes, connection relationships between nodes, and connection features corresponding to the connection relationships.

17. The search data processing device according to claim 16, characterized in that: The media search graph acquisition module is also used to acquire supplementary media data and generate a media node corresponding to the supplementary media data; the supplementary media data includes at least one of first media data and second media data, the first media data refers to media data with a media quality greater than a quality threshold, and the second media data refers to media data with a time interval between a published time and a current time less than a time interval threshold; and acquire at least one associated data corresponding to the supplementary media data.

18. The search data processing device according to claim 16, characterized in that: The media search graph acquisition module is also used to obtain the rewritten search term corresponding to the historical search term; when the search time interval between the historical search term and the corresponding rewritten search term is less than the time interval threshold, and the similarity between the historical search term and the corresponding rewritten search term is greater than the similarity threshold, a search term node corresponding to the rewritten search term is generated; and a connection relationship is established between the search term node corresponding to the historical search term and the search term node corresponding to the rewritten search term.

19. The search data processing device according to claim 15, characterized in that: The initial semantic feature determination module is further configured to input the sampling subgraphs of at least two semantic perspectives corresponding to the current node into the initial graph neural network to obtain the subgraph features of each sampling subgraph corresponding to the current node; The current node is a search term node or a media node in the first training sample pair, and different meta-paths correspond to different semantic perspectives; and the sub-graph features corresponding to the current node are fused to obtain the initial semantic features corresponding to the current node.

20. The search data processing device according to claim 19, characterized in that: The sampling subgraph corresponding to the node includes the node and the first-order neighbor nodes and second-order neighbor nodes corresponding to the node. The initial semantic feature determination module is further used to aggregate the node features corresponding to the second-order neighbor nodes in the current sampling subgraph corresponding to the current node and the connection features between the second-order neighbor nodes and the first-order neighbor nodes to the first-order neighbor nodes through the initial graph neural network to obtain the second-order aggregated features corresponding to the first-order neighbor nodes; Aggregating the node features, second-order aggregate features, and connection features between the first-order neighbor nodes and the current node to the current node through the initial graph neural network to obtain the first-order aggregate features corresponding to the current node; Through the initial graph neural network, based on the node features and first-order aggregation features corresponding to the current node, the subgraph features of the current sampling subgraph corresponding to the current node are obtained.

21. The search data processing device according to any one of claims 15 to 20, characterized in that: The network training module is further configured to obtain a node loss based on a difference between semantic feature pairs corresponding to the positive node pair and the negative node pair, respectively; obtain a plurality of second training sample pairs; the plurality of second training sample pairs include positive sampling subgraph pairs and negative sampling subgraph pairs, the positive sampling subgraph pairs include sampling subgraphs of different semantic perspectives corresponding to the same node, and the negative sampling subgraph pairs include sampling subgraphs corresponding to different nodes of the same type; input each sampling subgraph in the second training sample pairs into the initial graph neural network, obtain respective subgraph features of each sampling subgraph in the second training sample pairs, and form subgraph feature pairs corresponding to the second training sample pairs; Obtaining a perspective loss based on differences between sub-image feature pairs corresponding to the positive sampling sub-image pair and the negative sampling sub-image pair; Based on the node loss and the perspective loss, the initial graph neural network is trained to obtain a target graph neural network.

22. The search data processing device according to claim 21, characterized in that: The network training module is also used to obtain the semantic similarity corresponding to the semantic feature pair based on the feature similarity between the initial semantic features in the same semantic feature pair; fuse the semantic similarities corresponding to the same positive node pair and the corresponding negative node pairs to obtain the fused similarity corresponding to the positive node pair; the negative node pair corresponding to the positive node pair is a negative node pair that has overlapping nodes with the positive node pair; based on the difference between the semantic similarity corresponding to the same positive node pair and the fused similarity, obtain the node sub-loss corresponding to the positive node pair; based on the node sub-loss corresponding to each positive node pair, obtain the node loss.

23. The search data processing device according to claim 21, characterized in that: The network training module is also used to obtain the perspective similarity corresponding to the sub-image feature pair based on the feature similarity between the sub-image features in the same sub-image feature pair; fuse the perspective similarities corresponding to the same positive sampling sub-image pair and the corresponding negative sampling sub-image pairs to obtain the fused similarity corresponding to the positive sampling sub-image pair; obtain the perspective sub-loss corresponding to the positive sampling sub-image pair based on the difference between the perspective similarity corresponding to the same positive sampling sub-image pair and the fused similarity; and obtain the perspective loss based on the perspective sub-loss corresponding to each positive sampling sub-image pair.

24. A search data processing device, characterized in that: The device comprises: An image acquisition module is configured to acquire a sampling subgraph corresponding to a target node; the target node includes at least one of a search term node and a media node, and the sampling subgraph corresponding to the target node is sampled from a media search graph; A feature determination module is used to input the sampling subgraph corresponding to the target node into the target graph neural network to obtain the target semantic feature corresponding to the target node; the target semantic feature is used for data search; The training process of the target graph neural network includes the following steps: obtaining the media search graph; the media search graph includes a search term node corresponding to the search term, a media node corresponding to the media data, and an association node corresponding to the association data, wherein the association data includes data associated with at least one of the search term and the media data; obtaining a plurality of first training sample pairs from the media search graph; the plurality of first training sample pairs include positive node pairs and negative node pairs, the positive node pairs include search term nodes and media nodes connected to each other in the media search graph, and the negative node pairs include search term nodes and media nodes randomly combined in the media search graph ... randomly combined in the media search graph The current search meta-path is determined in the meta-path; the current search meta-path is a path formed by sequentially connecting the type flags corresponding to the search type, the first type and the second type respectively; the meta-path is a sampling path starting from the search term node or the media node in the media search graph; taking the current search term node as the search center node, sampling at least two nodes of the first type directly connected to the search center node from the media search graph as the first-order neighbor nodes corresponding to the search center node, sampling at least two nodes of the second type directly connected to the first-order neighbor node from the media search graph as the second-order neighbor nodes corresponding to the search center node, based on the search The central node and the corresponding first neighbor node and second neighbor node are used to obtain the sampling subgraph corresponding to the search central node under the current search meta-path; the current media meta-path is determined from at least one meta-path corresponding to the media data; the current media meta-path is a path formed by sequentially connecting the type flags corresponding to the media type, the third type and the fourth type respectively; with the current media node as the media central node, at least two nodes of the third type directly connected to the media central node are sampled from the media search graph as the first-order neighbor nodes corresponding to the media central node, and at least one node of the fourth type directly connected to the first-order neighbor node is sampled from the media search graph. Two nodes serve as the second-order neighbor nodes corresponding to the media center node, and based on the media center node and the corresponding first neighbor node and second neighbor node, the sampling subgraph corresponding to the media center node under the current media element path is obtained; the sampling subgraphs corresponding to the search term node and the media node in the first training sample pair are respectively input into the initial graph neural network to obtain the initial semantic features of the search term node and the media node in the first training sample pair, which constitute the semantic feature pairs corresponding to the first training sample pair; based on the differences between the semantic feature pairs corresponding to the positive node pair and the negative node pair, the initial graph neural network is trained to obtain the target graph neural network.

25. The search data processing device according to claim 24, characterized in that: The device also includes a data search module, which is further used to obtain the target semantic features corresponding to the current search data of the current search object as the current search features, and obtain the target semantic features corresponding to the candidate search data as the candidate search features; the current search data is the current search term or the current media data, and the candidate search data is the candidate search term or the candidate media data; based on the feature similarity between the current search feature and the candidate search feature, the target search data is determined from multiple candidate search data as the search result corresponding to the current search data.

26. The search data processing device according to claim 25, characterized in that: The data search module is further configured to obtain target semantic features corresponding to at least two pieces of forward media data corresponding to the current search object; Fusing the target semantic features corresponding to the at least two forward media data to obtain a target object feature corresponding to the current search object; Fusing the target object feature and the current search feature to obtain a search fusion feature; Based on the feature similarity between the search fusion feature and the candidate search feature, target search data is determined from a plurality of candidate search data as the search result corresponding to the current search data.

27. The search data processing device according to claim 25, characterized in that: The data search module is further configured to input candidate search features corresponding to each candidate search data into the first processing branch of the dual-tower search model to obtain first search features corresponding to each candidate search data; Inputting the current search feature into the second processing branch of the dual-tower search model to obtain a second search feature; inputting the first search feature and the second search feature into the output layer of the dual-tower search model to obtain a first matching degree between each candidate search data and the current search data; Based on the first matching degree, target search data is determined from the candidate search data as the search result corresponding to the current search data.

28. The search data processing device according to claim 27, characterized in that: The data search module is further configured to determine intermediate search data from the candidate search data based on the first matching degree, and use the target semantic feature corresponding to the intermediate search data as the intermediate search feature; Inputting the current search feature and the intermediate search feature into a single-tower search model to obtain a second matching degree between the intermediate search data and the current search data; Based on the second matching degree, target search data is determined from each intermediate search data as the search result corresponding to the current search data.

29. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 14 is implemented.

30. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 14 is implemented.

31. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 14 is implemented.

Images