Multimedia object detection method and device, storage medium and electronic equipment

Abstract

The invention discloses a multimedia object detection method and apparatus, a storage medium and an electronic device. The method comprises the steps of obtaining object isographic data and object heterogeneous graph data corresponding to a specified multimedia object; extracting first multimedia object data having an association relationship with a specified multimedia object from the object same composition data; extracting second multimedia object data having an association relationship with the specified multimedia object from the object heterogeneous graph data; and performing object identification according to the first multimedia object data and the second multimedia object data to determine a target multimedia object associated with the specified multimedia object. According to the technical scheme, in the object detection process, the object features, the incidence relation between the objects, the incidence relation between the objects and the media and other contents are fully considered, the scene limitation is thrown away from the features and the incidence relation of the objects, and therefore the coverage rate of multimedia object detection is increased.

Description

Technical Field

[0001] The present application belongs to the field of computer processing technology, and specifically relates to a detection method, device, storage medium and electronic device for multimedia objects. Background Art

[0002] With the development of Internet technology, social activities based on the Internet have become more and more popular in people's lives. Multimedia objects on social platforms are a form of identity for people on the Internet. However, since social activities conducted through the Internet are not face-to-face communication, it is difficult for people to perceive the true identity behind multimedia objects. Therefore, social activities based on multimedia objects have certain security risks. For example, some people may use multimedia objects to carry out improper activities. Therefore, accurately identifying such abnormal multimedia objects is conducive to maintaining network security.

[0003] Currently, distinguishing whether a multimedia object is abnormal is usually implemented based on a clustering algorithm, that is, clustering abnormal multimedia objects into one category for identification. However, this algorithm has a high degree of scene dependency, and different scenes require different clustering models for processing, resulting in a low coverage rate for multimedia object detection. Summary of the invention

[0004] The purpose of the present application is to provide a method, device, storage medium and electronic device for detecting multimedia objects, so as to improve the coverage of multimedia object detection.

[0005] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0006] According to one aspect of an embodiment of the present application, a method for detecting a multimedia object is provided, including:

[0007] Obtaining object isomorphic graph data and object heterogeneous graph data corresponding to a specified multimedia object, wherein the object isomorphic graph data is constructed based on associations between multimedia objects, and the object heterogeneous graph data is constructed based on associations between multimedia objects and object diffusion media, wherein the object diffusion media represents dimensions of similarity between multimedia objects;

[0008] Extracting first multimedia object data having an association relationship with the specified multimedia object from the object isomorphism graph data;

[0009] Extracting second multimedia object data having an association relationship with the specified multimedia object from the object heterogeneous graph data;

[0010] Object recognition is performed according to the first multimedia object data and the second multimedia object data to determine a target multimedia object associated with the designated multimedia object.

[0011] According to one aspect of an embodiment of the present application, a detection device for a multimedia object is provided, including:

[0012] A data acquisition module, used to acquire object isomorphic graph data and object heterogeneous graph data corresponding to a specified multimedia object, wherein the object isomorphic graph data is constructed based on the association relationship between multimedia objects, and the object heterogeneous graph data is constructed based on the association relationship between multimedia objects and object diffusion media, wherein the object diffusion media represents the dimension of similarity between multimedia objects;

[0013] A first data extraction module, configured to extract first multimedia object data associated with the specified multimedia object from the object isomorphism graph data;

[0014] A second data extraction module, configured to extract second multimedia object data associated with the specified multimedia object from the object heterogeneous graph data;

[0015] The object recognition module is used to perform object recognition according to the first multimedia object data and the second multimedia object data to determine a target multimedia object associated with the designated multimedia object.

[0016] In one embodiment of the present application, the object isomorphism graph data is constructed with multimedia objects as nodes and associations between multimedia objects as edges; the first data extraction module is specifically used for:

[0017] Query the object isomorphism graph data according to the designated multimedia object, and take the designated multimedia object and a multimedia object whose number of nodes between the designated multimedia object and the designated multimedia object is less than or equal to a first designated number as a first multimedia object;

[0018] The object feature data of the first multimedia objects and the feature data of the relationships between the first multimedia objects are used as first multimedia object data.

[0019] In one embodiment of the present application, the object heterogeneous graph data is constructed with multimedia objects and object diffusion media as nodes and with associations between multimedia objects and object diffusion media as edges; the second data extraction module is specifically used for:

[0020] Querying the object heterogeneous graph data according to the specified multimedia object to determine neighbor multimedia objects whose number of nodes separated from the specified multimedia object is less than or equal to a second specified number;

[0021] Generate relationship data between the specified multimedia object and the neighbor multimedia object according to the medium node data between the specified multimedia object and the neighbor multimedia object in the object isomorphism graph data; wherein the medium node data is used to represent the data of the node corresponding to the object diffusion medium;

[0022] The neighbor multimedia object and the designated multimedia object are used as second multimedia objects, and object feature data of the second multimedia objects and feature data of relationships between the second multimedia objects are used as second multimedia object data.

[0023] In one embodiment of the present application, the object recognition module includes:

[0024] A network construction unit, configured to generate a multimedia object network using the multimedia objects respectively included in the first multimedia object data and the second multimedia object data as nodes;

[0025] The object recognition unit is used to perform object recognition according to the multimedia object network to determine a target multimedia object associated with the designated multimedia object in the multimedia object network.

[0026] In one embodiment of the present application, the object recognition unit includes:

[0027] The classification processing subunit is used to classify the multimedia objects in the multimedia object network to obtain the classification results of the multimedia objects in the multimedia object network; and determine the target multimedia object associated with the specified multimedia object in the multimedia object network according to the classification results.

[0028] In one embodiment of the present application, the classification processing subunit is specifically used for:

[0029] Generate a first object relationship matrix according to the association relationship between the multimedia objects included in the multimedia object network;

[0030] generating an object feature matrix according to object feature data of multimedia objects contained in the multimedia object network;

[0031] Feature extraction and mapping processing are performed according to the first object relationship matrix and the object feature matrix to obtain classification results of multimedia objects in the multimedia object network.

[0032] In one embodiment of the present application, the classification processing subunit is specifically used for:

[0033] Extract the neighbor features of the multimedia objects in the multimedia object network according to the first object relationship matrix, and extract the object feature data of the multimedia objects in the multimedia object network according to the object feature matrix; the neighbor features of the multimedia objects include the object feature data of other multimedia objects associated with the multimedia object.

[0034] Fuse the neighbor information of the multimedia object and the node feature information to obtain the fused feature of the multimedia object.

[0035] Concatenate the object feature data of the multimedia object and the fused feature of the multimedia object to obtain the feature to be classified of the multimedia object.

[0036] Perform mapping processing according to the feature to be classified of the multimedia object to obtain the classification result of the multimedia object.

[0037] In an embodiment of the present application, the classification processing subunit is specifically configured to:

[0038] Sample the multimedia objects in the multimedia object network to obtain a plurality of multimedia objects to be recognized.

[0039] Generate a relationship matrix of objects to be recognized according to the association relationships between the plurality of multimedia objects to be recognized in the multimedia object network, and generate a feature matrix of objects to be recognized according to the object feature data of the plurality of multimedia objects to be recognized in the multimedia object network.

[0040] Perform feature extraction and mapping processing according to the relationship matrix of objects to be recognized and the feature matrix of objects to be recognized to obtain the classification results of the multimedia objects to be recognized in the multimedia object network.

[0041] In an embodiment of the present application, the classification processing subunit is specifically configured to:

[0042] Sample the multimedia objects in the multimedia object network according to the type of object diffusion medium involved in establishing the association relationship between the multimedia objects in the multimedia object network and the specified multimedia object, and the sampling weight corresponding to the type of object diffusion medium, to obtain a plurality of multimedia objects to be recognized.

[0043] In an embodiment of the present application, the object recognition unit includes:

[0044] The similarity recognition subunit is used to calculate the similarity between the multimedia objects in the multimedia object network and the specified multimedia object, and obtain the similarity results of the multimedia objects in the multimedia object network; and determine the target multimedia object associated with the specified multimedia object in the multimedia object network according to the similarity results.

[0045] In one embodiment of the present application, the similarity identification subunit is specifically used for:

[0046] Generate a second object relationship matrix according to the association relationships and relationship weights between multimedia objects in the multimedia object network;

[0047] generating a similarity matrix between multimedia objects according to object feature data of multimedia objects in the multimedia object network;

[0048] generating an object node degree matrix according to the number of other multimedia objects directly connected to the multimedia object in the multimedia object network;

[0049] The similarity results of multimedia objects in the multimedia object network are calculated according to the second object relationship matrix, the similarity matrix and the object node degree matrix.

[0050] In one embodiment of the present application, the similarity identification subunit is specifically used for:

[0051] Multiplying the second object relationship matrix by the similarity matrix to obtain a first matrix product, and multiplying the first matrix product by a first weight to obtain a first calculation result;

[0052] Multiplying the object node degree matrix and the object position matrix to obtain a second matrix product, and multiplying the second matrix product and a second weight to obtain a second calculation result; wherein the object position matrix represents the position of the specified multimedia object in the multimedia object network, and the sum of the first weight and the second weight is 1;

[0053] The first calculation result and the second calculation result are combined to obtain a candidate similarity result;

[0054] When the difference between the candidate similarity result and the similarity result calculated last time is greater than a preset threshold, taking the candidate similarity result as a new similarity matrix, and recalculating the candidate similarity result based on the new similarity matrix;

[0055] When the difference between the candidate similarity result and the similarity result obtained by the previous calculation is less than or equal to the preset threshold, the candidate similarity result is used as the similarity result.

[0056] In one embodiment of the present application, the object recognition unit is specifically used to:

[0057] Classifying the multimedia objects in the multimedia object network to obtain classification results of the multimedia objects in the multimedia object network;

[0058] Determine, according to the classification result, a first target multimedia object in the multimedia object network that is of the same type as the designated multimedia object;

[0059] Calculating the similarity between the multimedia objects in the multimedia object network and the specified multimedia object to obtain a similarity result of the multimedia objects in the multimedia object network;

[0060] Determine, according to the similarity result, a second target multimedia object in the multimedia object network whose similarity to the designated multimedia object is greater than a preset threshold;

[0061] A target multimedia object associated with the designated multimedia object in the multimedia object network is generated according to the first target multimedia object and the second target multimedia object.

[0062] According to one aspect of an embodiment of the present application, a computer-readable medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for detecting multimedia objects in the above technical solution is implemented.

[0063] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor executes the executable instructions so that the electronic device performs a method for detecting multimedia objects as in the above technical solution.

[0064] According to one aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the detection method of the multimedia object in the above technical solution.

[0065] In the technical solution provided in the embodiment of the present application, object isomorphic graph data is constructed according to the association relationship between multimedia objects, and object heterogeneous graph data is constructed according to the association relationship between multimedia objects and object diffusion media. Based on these two types of graph data, first multimedia object data and second multimedia object data that have an association relationship with the specified multimedia object are respectively extracted, and then object recognition is performed based on these two types of data to detect the target multimedia object related to the specified multimedia object. In the object detection process, multiple aspects such as object characteristics, association relationships between objects, and association relationships between objects and media are fully considered. Starting from the characteristics and association relationships of the objects themselves, the limitations of the scene are discarded, thereby improving the coverage of multimedia object detection.

[0066] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0068] Figure 1 An exemplary system architecture block diagram applying the technical solution of the present application is schematically shown.

[0069] Figure 2 A flowchart of a multimedia object detection method provided by an embodiment of the present application is schematically shown.

[0070] Figure 3 The object isomorphic graph data and a schematic diagram of the object isomorphic graph data provided by one embodiment of the present application are schematically shown.

[0071] Figure 4 A schematic diagram schematically shows a method for detecting a multimedia object provided by an embodiment of the present application.

[0072] Figure 5 A schematic diagram schematically shows a training process of a classification model provided by an embodiment of the present application.

[0073] Figure 6 A flowchart of a multimedia object detection method provided by an embodiment of the present application is schematically shown.

[0074] Figure 7A-7B A flowchart of a method for detecting multimedia objects based on similarity provided by an embodiment of the present application is schematically shown.

[0075] Figure 8 The structural block diagram of the multimedia object detection device provided in the embodiment of the present application is schematically shown.

[0076] Fig. 9 The structure block diagram of a computer system suitable for implementing an electronic device of an embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0077] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0078] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0079] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0080] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0081] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0082] It can be understood that in the specific implementation of the present application, related data such as object information (such as object creation time, creation device, etc.) is involved. When the above embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0083] Figure 1 The exemplary system architecture block diagram applying the technical solution of the present application is schematically shown.

[0084] like Figure 1 As shown, the system architecture 100 may include a terminal device 110, a network 120 and a server 130. The terminal device 110 may include a smart phone, a tablet computer, a laptop computer, an intelligent voice interaction device, a smart home appliance, a vehicle-mounted terminal, etc. The server 130 may be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The network 120 may be a communication medium of various connection types that can provide a communication link between the terminal device 110 and the server 130, for example, a wired communication link or a wireless communication link.

[0085] According to the implementation requirements, the system architecture in the embodiment of the present application can have any number of terminal devices, networks and servers. For example, the server 130 can be a server group composed of multiple server devices. In addition, the technical solution provided in the embodiment of the present application can be applied to the terminal device 110, can also be applied to the server 130, or can be implemented by the terminal device 110 and the server 130 together, and the present application does not make any special restrictions on this.

[0086] In one embodiment of the present application, the technical solution of the present application is implemented by the server 130. The server 130 obtains object isomorphic graph data and object heterogeneous graph data corresponding to a specified multimedia object. The object isomorphic graph data is constructed based on the association relationship between multimedia objects. The object heterogeneous graph data is constructed based on the association relationship between multimedia objects and object diffusion media. The object diffusion media represents the dimension of similarity between multimedia objects. Figure 1In the system architecture shown, a user can create a multimedia object through a terminal device 110, a terminal device 110 can create multiple multimedia objects, and a multimedia object can be used on multiple different terminal devices 110. Assuming that multimedia object A and multimedia object B are both used on device A, then multimedia object A and multimedia object B are the same on the device in use, that is, device A can serve as a diffusion medium between multimedia object A and multimedia object B. Next, the server 130 extracts the first multimedia object data that has an association relationship with the specified multimedia object from the object isomorphic graph data, and extracts the second multimedia object data that has an association relationship with the specified multimedia object from the object heterogeneous graph data. Finally, the server 130 performs object recognition based on the first multimedia object data and the second multimedia object data to determine the target multimedia object associated with the specified multimedia object. For example, the specified multimedia object is an improper activity object. By implementing the technical solution of the present application, the server 130 can find other improper activity objects or potential improper activity objects that are associated with the improper activity object, and subsequently these objects can be subjected to risk control detection to improve the security of the network platform.

[0087] The following is a detailed description of the multimedia object detection method provided by the present application in conjunction with specific implementation methods.

[0088] Figure 2 The flowchart of a method for detecting a multimedia object provided by an embodiment of the present application is schematically shown. The method can be implemented by a terminal device or a server. Figure 2 As shown, the multimedia object detection method provided in this embodiment includes steps 210 to 240, which are specifically as follows:

[0089] Step 210: Obtain object isomorphic graph data and object heterogeneous graph data corresponding to the specified multimedia object. The object isomorphic graph data is constructed based on the association relationship between multimedia objects. The object heterogeneous graph data is constructed based on the association relationship between multimedia objects and object diffusion media. The object diffusion media represents the dimension of similarity between multimedia objects.

[0090] Specifically, multimedia objects are objects generated based on multimedia technology. For example, a media object can be a multimedia account in a multimedia platform, such as an account in a social platform. Object isomorphism graph data is constructed based on the association relationship between multimedia objects. The association relationship between multimedia objects can reflect the elements through which different multimedia objects are connected. For example, if two multimedia objects are created at the same time, then the association relationship between the two multimedia objects means that the two multimedia objects are created at the same time. It can be understood that there can be many types of association relationships between multimedia objects, such as the same creation time, reference to the same activity, being in a certain group, etc.

[0091] The object heterogeneous graph data reflects the association between multimedia objects and object diffusion media. In the literal sense, diffusion media refers to a substance that can diffuse certain information so that other objects can also be infected with the information. For example, sound can be transmitted from one location to another through the air, and air is the diffusion medium of sound. Object diffusion medium refers to the ability to diffuse certain characteristics of multimedia objects so that different multimedia objects have similar characteristics. Therefore, object extension medium can characterize the similarity dimension between multimedia objects. Object extension medium usually belongs to substances with practical meaning, such as hardware devices, IP (Internet Protocol) addresses, specific certificates, WiFi (Wireless Fidelity) networks, etc. When the use of a multimedia object depends on a certain object diffusion medium, the multimedia object has an association relationship with the object diffusion medium. For example, if a multimedia object is created on a certain terminal device, the multimedia object has an association relationship with the terminal device. For another example, if the IP address of a multimedia object when performing network activities is IP1, the multimedia object has an association relationship with IP1. Obviously, one multimedia object may be associated with multiple object diffusion media, and one object diffusion medium may also be associated with multiple multimedia objects.

[0092] It can be understood that whether it is object homogeneous graph data or object heterogeneous graph data, it should include object feature data of the multimedia object itself, such as the object name, object identifier, object description information, etc. of the multimedia object.

[0093] In one embodiment of the present application, object isomorphic graph data is constructed in the form of a graph database, which is a relational database including multiple nodes and the relationship between nodes. In the object isomorphic graph data, the nodes are multimedia objects, and the relationship between the nodes is reflected by the edge. When two multimedia objects have a certain association relationship, the two multimedia objects are connected to form an edge. Then, when it is detected that the two multimedia objects are connected by an edge, it can be determined that the two multimedia objects have a certain association relationship. Similarly, object heterogeneous graph data is also constructed in the form of a graph database, and the nodes include multimedia objects and object diffusion media. In order to distinguish which type of node a node belongs to, the node type can be marked in the object feature data of the node, that is, the node is marked as an object node or a medium node. Further, for medium nodes, classification can be continued, such as device nodes, IP nodes, WiFi nodes, etc. The association relationship between multimedia objects and object diffusion media is reflected by the edge between nodes. If a multimedia object has an association relationship with a certain object diffusion medium, an edge is constructed between the multimedia object and the object diffusion medium.

[0094] Exemplarily, taking the multimedia object as a media account as an example, the object isomorphic graph is recorded as the account isomorphic graph, and the object heterogeneous graph is recorded as the account heterogeneous graph. Figure 3 The following schematically shows the account isomorphic graph data and the account isomorphic graph data provided by one embodiment of the present application. Figure 3 As shown, the account isomorphic graph data reflects the association relationship between multimedia accounts, which can be referred to as "account-account" data, that is, the nodes are all multimedia accounts. In the account isomorphic graph data, the feature data of the edge includes time window, constraints and similarity. The similarity refers to the similarity between the two nodes (i.e., two multimedia accounts) calculated based on the account feature data of the nodes connected at both ends of the edge when the two parameters of time window and constraint are the same; the time window refers to a certain time period, for example, time window 1 can refer to 8:00-9:00; the constraints are used to limit the fixed conditions when calculating the similarity, for example, IPBlack means that the similarity is calculated under the condition that the two accounts belong to the same IP address, DeviceType means that the similarity is calculated under the condition that the device type is the same, and MsgType means that the similarity is calculated when the same information is published. Assume that Figure 3 The first piece of data in the account isomorphism graph data is the feature data of edge 1 between account A and account B. This means that, under time window 1 (for example, 8:00-9:00) and constraint condition DeviceType, based on the account feature data of account A and account B (such as the number of information published on the multimedia platform, the type of information clicked, etc.), the similarity between account A and account B is calculated to be 0.5.

[0095] like Figure 3 As shown, the account heterogeneous graph data reflects the association between multimedia accounts and account diffusion media, which can be referred to as "account-media" data, that is, the nodes include multimedia accounts and account diffusion media. The content recorded in the account heterogeneous graph data includes multimedia accounts (referred to as accounts), media types and media content. Multimedia accounts are represented by account identifiers, such as 001; media types refer to the types of account diffusion media, such as device, IP (IP address), ID card, etc.; media content is a specific description of the account diffusion medium, for example, the media content of device can be a specific device identifier, the media content of IP can be a specific IP address, and the media content of ID card can be a specific ID card number.

[0096] In one embodiment of the present application, the designated multimedia object is a multimedia object of a determined type. For example, the designated multimedia object may be a multimedia object that has been determined to be abnormal, such as a multimedia object that performs inappropriate activities.

[0097] Step 220: extract first multimedia object data associated with the designated multimedia object from the object isomorphism graph data.

[0098] Specifically, the object isomorphism graph data is queried according to the specified multimedia object to obtain first multimedia object data having an association relationship with the specified multimedia object. When a multimedia object has an association relationship with the specified multimedia object, it is connected to the specified multimedia object through an edge. Therefore, the multimedia object having an edge connection with the specified multimedia object in the object isomorphism graph data can be extracted as the first multimedia object data.

[0099] In one embodiment of the present application, the process of extracting specified first multimedia object data includes: querying object isomorphism graph data based on the specified multimedia object, taking the specified multimedia object and the multimedia objects whose number of nodes separated from the specified multimedia object is less than or equal to a first specified number as the first multimedia object; taking the object feature data of the first multimedia object and the relationship feature data between the first multimedia objects as the first multimedia object data.

[0100] Specifically, other multimedia objects that have an association relationship with the specified multimedia object are called neighbor multimedia objects of the specified multimedia object. In the object isomorphism graph data, when a neighbor multimedia object is directly adjacent to the specified multimedia object through an edge, the neighbor multimedia object is called a first-order neighbor of the specified multimedia object. Similarly, when a neighbor multimedia object is connected to the specified multimedia object through N-1 nodes, the neighbor multimedia object is called an N-order neighbor of the specified multimedia object. When extracting the first multimedia object data, the corresponding multimedia objects within the specified order neighbors of the specified multimedia object can be extracted as the multimedia objects in the first multimedia object data, that is, the first multimedia object whose number of nodes between the specified multimedia object and the specified multimedia object is less than or equal to the first specified number is extracted. Exemplarily, assuming that the first specified number is 1, then the extracted first multimedia object is separated from the specified multimedia object by at most 1 node, that is, the extracted multimedia objects are within the second-order neighbors of the specified multimedia object, then including the second-order neighbor multimedia objects and the first-order neighbor multimedia objects corresponding to the specified multimedia object. Therefore, the first multimedia object data is actually centered on the designated multimedia object, and other multimedia objects are directly connected to the designated multimedia object or connected through a node. Of course, other multimedia objects can also be connected to each other, and other multimedia objects can be first-order, second-order or even N-order neighbors.

[0101] The extracted multimedia objects and the specified multimedia objects are collectively referred to as first multimedia objects. The object feature data of each first multimedia object in the object isomorphism graph data and the relationship feature data between each first multimedia object are the first multimedia object data. The object feature data of the first multimedia object is equivalent to the node data itself in the object isomorphism graph data, and the relationship feature data between the first multimedia objects is equivalent to the edge data between the nodes in the object isomorphism graph data.

[0102] Step 230: extract second multimedia object data associated with the designated multimedia object from the object heterogeneous graph data.

[0103] Specifically, the process of extracting the second multimedia object data from the object heterogeneous graph data is similar to the process of extracting the first multimedia object data from the object homogeneous graph data, that is, the node data that constructs the edge with the specified multimedia object and the corresponding edge data can be extracted as the second multimedia object data.

[0104] In one embodiment of the present application, the extraction process of the second multimedia object data includes: querying the object heterogeneous graph data according to the specified multimedia object, and determining the neighbor multimedia objects whose number of nodes separated from the specified multimedia object is less than or equal to the second specified number; generating the relationship data between the specified multimedia object and the neighbor multimedia objects according to the medium node data between the specified multimedia object and the neighbor multimedia objects in the object isomorphic graph data; taking the neighbor multimedia objects and the specified multimedia objects as the second multimedia objects, and taking the object feature data of the second multimedia objects and the relationship feature data between the second multimedia objects as the second multimedia object data.

[0105] Specifically, the nodes in the object heterogeneous graph data include multimedia object nodes (referred to as object nodes for short) and object diffusion medium nodes (referred to as medium nodes for short). Since the application ultimately wants to detect multimedia objects, when extracting object data, it is necessary to find object nodes based on medium nodes. In the object heterogeneous graph data, object nodes are generally directly connected to medium nodes. Therefore, to extract object nodes based on specified multimedia objects, it is often necessary to extract multi-order neighbor nodes of the specified multimedia object. In this embodiment, the order of the extracted multi-order neighbor nodes is limited by the second specified number. Generally, the second specified number is greater than the first specified number. According to the definition of neighbor nodes, when a multimedia object is separated from a specified multimedia object by a second specified number of medium nodes, the multimedia object is called the [second specified number + 1]th order neighbor of the specified multimedia object, and the neighbor multimedia objects within the [second specified number + 1]th order neighbor range of the specified multimedia object are extracted as the required neighbor multimedia objects. Exemplarily, assuming that the second specified number is 3, then the second multimedia object is a multimedia object within the fourth-order neighbor range of the specified multimedia object. For example, assuming that the object heterogeneous graph data has the following connection relationship, object A-device x-IP1-WiFi1-object B, where object A is the specified multimedia object, then there are 3 media nodes between object B and object A, and object B is the fourth-order neighbor of object A, which is used as the required neighbor multimedia object; if the connection relationship is: object A-device x-object B-IP1-WiFi1-object C, then object B is the second-order neighbor of object A and can be used as the required neighbor multimedia object, and object C is the fifth-order neighbor of object A and cannot be used as the required neighbor multimedia object.

[0106] After determining the neighbor multimedia objects, the media node data between the designated multimedia object and the neighbor multimedia objects actually reflects the association relationship between the designated multimedia object and the neighbor multimedia objects. Therefore, the relationship data between the designated multimedia object and the neighbor multimedia objects is generated based on the media node data between the designated multimedia object and the neighbor multimedia objects. For example, for the connection relationship "object A-device x-object B", the relationship data generated between object A and object B is: object A and object B are connected through device x. Optionally, the media node data can also be added to the object node as the object feature data of the object node itself. For example, device x is added to the object feature data of object A, and at the same time, device x is added to the object feature data of object B. Since both object feature data contain device x, it can be determined that the two objects are connected through device x.

[0107] The extracted neighbor multimedia objects and the designated multimedia objects are collectively referred to as second multimedia objects. The object feature data of each second multimedia object in the object heterogeneous composition data and the relationship feature data between each second multimedia object are the second multimedia object data, wherein the relationship feature data between the second multimedia objects include the relationship feature data generated based on the media node data and the original relationship feature data.

[0108] Step 240: Perform object recognition according to the first multimedia object data and the second multimedia object data to determine a target multimedia object associated with the designated multimedia object.

[0109] Specifically, in obtaining the first multimedia object data and the second multimedia object data, it is equivalent to obtaining a plurality of multimedia objects associated with the designated multimedia object from the object level and the medium level, and then performing object recognition on these multimedia objects to find the target multimedia object associated with the designated multimedia object. Object recognition can be achieved through object classification, so the target multimedia object can be a multimedia object of the same type as the designated multimedia object. Object recognition can also be achieved through similarity calculation, so the target multimedia object can be a multimedia object with a high similarity to the designated multimedia object.

[0110] In one embodiment of the present application, when performing object recognition, a multimedia object network can be first generated based on the first multimedia object data and the second multimedia object data. The multimedia object network only uses multimedia objects as nodes and the association relationships between multimedia objects as edges. By constructing the multimedia object network, the first multimedia object data and the second multimedia object data can be integrated.

[0111] And remove duplicate items in the two data. For example, if both the first multimedia object data and the second multimedia object data include a specified multimedia object, when constructing a multimedia object network, only one node is used to represent the specified multimedia object. That is, when the same multimedia object exists in the two data, one node should be used to represent the multimedia object. Then, object recognition is performed based on the multimedia object network to determine the target multimedia object associated with the specified multimedia object in the multimedia object network.

[0112] In the technical solution provided in the embodiment of the present application, object isomorphic graph data is constructed according to the association relationship between multimedia objects, and object heterogeneous graph data is constructed according to the association relationship between multimedia objects and object diffusion media. Based on these two types of graph data, first multimedia object data and second multimedia object data that have an association relationship with the specified multimedia object are respectively extracted, and then object recognition is performed based on these two types of data to detect the target multimedia object related to the specified multimedia object. In the object detection process, multiple aspects such as object characteristics, association relationships between objects, and association relationships between objects and media are fully considered. Starting from the characteristics and association relationships of the objects themselves, the limitations of the scene are discarded, thereby improving the coverage of multimedia object detection.

[0113] For example, Figure 3 A schematic diagram schematically shows a method for detecting a multimedia account provided by an embodiment of the present application. Figure 3 In the method shown, multimedia objects are media accounts, object isomorphic graphs are recorded as account isomorphic graphs, and object heterogeneous graphs are recorded as account heterogeneous graphs. Figure 3 As shown, the method includes a relationship pair construction process, a database query process and a diffusion detection process.

[0114] Relationship pair construction process: Construct account isomorphic graph data based on the association relationship between multimedia accounts, that is, Figure 3 The “account-account” data shown in FIG. 1 is a graph of the “account-account” data. At the same time, the account heterogeneous graph data is constructed based on the association relationship between the multimedia account and the account diffusion medium, that is, Figure 3 The "account-medium" data shown. The specific contents of the account homogeneous graph data and the account heterogeneous graph data can refer to the relevant description in the aforementioned step 210, which will not be repeated here.

[0115] Database query process: First, the account isomorphic graph data and the account heterogeneous graph data are stored in the database as data in the graph database. Then query in the graph database according to the black seed, where the "black seed" is the designated multimedia account in the aforementioned embodiment, and is also an abnormal multimedia account. "Black seeds" can be obtained through manual review, quality inspection, improper permanent blocking, channel feedback, etc., where manual review is manual review of multimedia accounts to determine whether they are abnormal multimedia accounts. Improper permanent blocking refers to selecting one of the identified multimedia accounts with improper activities as a black seed. Quality inspection is to perform quality inspection on multimedia accounts to identify abnormal multimedia accounts. Channel feedback refers to selecting black seeds from information on abnormal multimedia accounts received from other channels.

[0116] Diffusion detection: According to the query results of the graph database, the first multimedia account data and the second multimedia account data can be obtained. The specific query process can refer to the relevant description of steps 220-230 in the above embodiment, which will not be repeated here. The first multimedia account data and the second multimedia account data are actually account data extracted from the multi-order neighbors of the black seed, that is, the multi-order diffusion of the black seed in the graph database, and the other nodes obtained after the multi-order diffusion are equivalent to the nodes in the community formed around the black seed. Diffusion detection is performed in the local community to obtain the detection results and determine the target multimedia account associated with the black seed.

[0117] Figure 4 The following schematically shows a method for detecting multimedia objects provided by an embodiment of the present application. This embodiment is a further refinement of the above embodiment. Figure 4 As shown, the multimedia object detection method provided in the embodiment of the present application includes steps 410 to 460, which are specifically as follows:

[0118] Step 410: Obtain object isomorphic graph data and object heterogeneous graph data corresponding to the specified multimedia object. The object isomorphic graph data is constructed based on the association relationship between multimedia objects. The object heterogeneous graph data is constructed based on the association relationship between multimedia objects and object diffusion media. The object diffusion media represents the dimension of similarity between multimedia objects.

[0119] Step 420: extract first multimedia object data associated with the designated multimedia object from the object isomorphism graph data.

[0120] Step 430: extract second multimedia object data associated with the designated multimedia object from the object heterogeneous graph data.

[0121] Step 440: Generate a multimedia object network using the multimedia objects respectively included in the first multimedia object data and the second multimedia object data as nodes.

[0122] Steps 410 - 440 can be seen from the relevant descriptions of steps 210 - 240 in the aforementioned embodiment, which will not be repeated here.

[0123] In one embodiment of the present application, the multimedia object network also includes the weight of the edge, and the weight of the edge reflects the strength of the object diffusion medium for the object diffusion, or the importance of the object similarity. For example, for a device, if two multimedia objects are created based on the same device or used on the same device, the two multimedia objects are more likely to be multimedia objects of the same type or with high similarity, which means that the device has a strong object diffusion strength. For IP addresses, in a mobile data network, IP addresses are usually assigned to devices by base stations. The same device may use different IP addresses, and different devices may use the same IP address. Therefore, the probability that two multimedia objects use the same IP address and the corresponding two multimedia objects are associated is lower than the probability that two media objects are used on the same device and the corresponding two multimedia objects are associated, which means that the diffusion strength of the IP address is lower than the diffusion strength of the device. Therefore, when constructing an edge in a multimedia object network, if the two multimedia objects connected at both ends of the edge are connected based on an object diffusion medium with a large diffusion strength (for example, connected by a device), then the weight of the edge can be set to be larger; if the two multimedia objects connected at both ends of the edge are connected based on an object diffusion medium with a small diffusion strength (for example, connected by an IP address, WiFi, etc.), then the weight of the edge can be set to be smaller.

[0124] Step 450: Classify the multimedia objects in the multimedia object network to obtain classification results of the multimedia objects in the multimedia object network.

[0125] Specifically, a multimedia object network is a network constructed with multimedia objects as nodes and the relationships between nodes as edges. The network can also be regarded as a graph composed of multiple nodes. Therefore, the multimedia objects in the multimedia object network can be classified through a classification model based on a graph convolutional neural network.

[0126] In one embodiment of the present application, the classification processing process specifically includes: generating a first object relationship matrix based on the association relationship between multimedia objects contained in the multimedia object network; generating an object feature matrix based on the object feature data of the multimedia objects contained in the multimedia object network; performing feature extraction and mapping processing based on the first object relationship matrix and the object feature matrix to obtain the classification results of the multimedia objects in the multimedia object network.

[0127] Specifically, two matrices are first generated according to the multimedia object network: a first object relationship matrix and an object feature matrix. The first object relationship matrix reflects the association relationship between multimedia objects in the multimedia object network, that is, the relationship between nodes. Exemplarily, the first object relationship matrix includes multiple rows and multiple columns, and the elements corresponding to the i-th row and the j-th column represent the association relationship between the i-th node and the j-th node. If there is an association relationship between the two nodes, that is, the two nodes are directly connected by an edge, then the elements corresponding to the two nodes in the first object relationship matrix are set to 1; if there is no association relationship between the two nodes, that is, there is no edge between the two nodes, then the elements corresponding to the two nodes in the first object relationship matrix are set to 0. The object feature matrix is ​​composed of feature data of multimedia objects in the multimedia object network, and can also be called a node feature matrix, that is, the elements in the object feature matrix represent the object feature data of the node, such as whether the object is newly registered, the registration address of the object, the number of registration days, etc. It can be understood that the object feature data of the node often has multiple types, so the node feature matrix is ​​a multidimensional matrix.

[0128] After obtaining the first object relationship matrix and the object feature matrix, the two matrices are subjected to feature extraction and mapping processing to obtain the classification result of the multimedia object. This process can be implemented by a classification model based on a graph convolutional neural network. In this process, for each multimedia object, the neighbor nodes of the multimedia object, that is, the nodes with an association relationship with the multimedia object, are determined from the first object relationship matrix, and then the feature information of the neighbor nodes is extracted. At the same time, the node data of the multimedia object itself, that is, its own object feature data, is extracted from the object feature matrix. Then, the neighbor node feature information of the multimedia object and its own object feature data are fused and aligned. The two data can be fused by using average pooling (Mean), maximum pooling (Max), LSTM (Long Short-Term Memory) network or attention mechanism. The fusion process is actually to perform cross calculations such as dot product and multiplication on the two data to obtain the fusion feature of the multimedia object. Next, the object feature data of the multimedia object itself and the fused fusion feature are spliced ​​as the feature to be classified of the multimedia object node. Optionally, after splicing, the spliced ​​features can be subjected to nonlinear transformation, such as softmax, rule function, etc., to obtain the feature to be classified. Finally, the features to be classified are mapped, and the mapping process can be implemented by the softmax function to obtain the classification results of the multimedia objects.

[0129] In one embodiment of the present application, the classification processing process can also be: sampling multimedia objects in a multimedia object network to obtain multiple multimedia objects to be identified; generating an object relationship matrix to be identified based on the association relationship between the multiple multimedia objects to be identified in the multimedia object network, and generating an object feature matrix to be identified based on object feature data of the multiple multimedia objects to be identified in the multimedia object network; performing feature extraction and mapping processing based on the object relationship matrix to be identified and the object feature matrix to be identified to obtain classification results of the multimedia objects to be identified in the multimedia object network.

[0130] That is, before the classification process is performed, the multimedia objects in the multimedia object network are sampled, and a plurality of multimedia objects to be identified are obtained based on the sampling, and subsequently only the plurality of multimedia objects to be identified need to be classified. This is because the multimedia object network obtained according to the specified multimedia object includes a large amount of multimedia object data, and some multimedia objects have an indirect association relationship with the specified multimedia object, such as being connected through multiple object extension media. Such multimedia objects actually have a weak association relationship with the specified multimedia object and are unlikely to belong to the same type of object as the specified multimedia object. Therefore, objects with weak association relationships are eliminated through sampling, and multimedia objects with strong association relationships are subsequently detected, which ensures both the accuracy and efficiency of multimedia object detection. The generation process of the object relationship matrix to be identified and the object feature matrix to be identified are similar to the generation process of the first object relationship matrix and the object feature matrix, respectively. The subsequent feature extraction and mapping process are also similar to the previous text, and will not be repeated here.

[0131] In one embodiment of the present application, when sampling, different sampling weights can be set based on different object diffusion media, so that neighbors with strong medium association are sampled as much as possible, that is, multimedia objects that are associated with the specified multimedia object through the object extension medium with high diffusion strength are sampled as much as possible. Therefore, the sampling weight can be set according to the object diffusion medium type, and then the multimedia objects in the multimedia object network are sampled according to the object diffusion medium type involved in establishing an association relationship between the multimedia objects in the multimedia object network and the specified multimedia object, and the sampling weight corresponding to the object diffusion medium type, to obtain multiple multimedia objects to be identified. For example, all multimedia objects connected to the specified multimedia object through the device are sampled, and 80% of the multimedia objects connected to the specified multimedia object through the IP address are sampled.

[0132] Optionally, you can also set hyperparameters to make sampling pay more attention to the multimedia objects corresponding to one or more object extension media, and sample the same number of multimedia objects for these object diffusion media. For example, set to focus on multimedia objects connected to a certain device. Optionally, you can also sample based on the order of neighbors between multimedia objects and specified multimedia objects. Neighbors with smaller orders usually have a greater correlation with the specified multimedia object, and neighbors with larger orders have a smaller correlation with the specified multimedia object. For example, first-order neighbors have a greater correlation than second-order neighbors. Therefore, you can set the number of samples for neighbors with smaller orders to be higher than that for neighbors with larger orders. That is, the higher the order, the more cautious the sampling strategy.

[0133] In one embodiment of the present application, the classification model is obtained by pre-training with sample data. After the object isomorphic graph data and the object heterogeneous graph data are stored in the database, the multimedia objects of the identified type in the graph database can be used as sample data for model training. The basic information of the nodes in the graph database may include node labels, node types, node multidimensional features, node entry time, etc. The node label is used to mark which type the multimedia object belongs to, for example, marking the node as a positive sample or a negative sample, or marking which object type the node belongs to. The node type is used to mark whether the node is an object node or a medium node. The medium node can be further divided according to the medium type, such as device, IP, etc. The node multidimensional feature is a multiple feature vector, such as object registration time, registration address, etc. The node entry time is the time when the node is stored in the graph database. The basic information of edges in the graph database may include the starting node of the edge, the ending node of the edge, the edge type, the multidimensional features of the edge, and the storage time of the edge. The type of edge is equivalent to the relationship type between nodes, such as device connection, IP connection, constraint connection, etc. The multidimensional features of the edge are used to record the information of the edge. For example, the edge corresponding to the device records whether the device is a new device, and the edge corresponding to the IP records the specific area of ​​the IP. The multidimensional features of the edge can also include the weight of the edge, the update time of the edge, etc. The storage time of the edge is also a multidimensional feature of the edge.

[0134] During model training, data can be obtained from the graph database based on the specified sample objects to construct a sample object network, and then sample multimedia objects are sampled according to the sample object network, and a sample object relationship matrix and a sample object feature matrix are generated according to the sampling results, and then feature extraction and mapping processing are performed according to the sample object relationship matrix and the sample object feature matrix. This part is the same as the aforementioned process of feature extraction and mapping processing of the first object relationship matrix and the object feature matrix, and will not be repeated. After constructing the object network, the training set and the test set can be divided according to the set ratio, and the model can be trained with the data of the training set, and the trained classification model can be tested with the data of the test set.

[0135] For example, Figure 5 The following is a schematic diagram of the training process of a classification model provided by an embodiment of the present application. Figure 5 As shown in Figure 1, the training process of the classification model includes three stages: sampling neighbor information, fusing neighbor information, and training & prediction.

[0136] Sampling neighbor information: query the graph database with the black seed (specified sample object) to obtain multiple neighbor objects. The first-degree neighbor means the node directly connected to the black seed, and the second-degree neighbor means the node connected to the black seed after one node. The sampling process is based on the description in the previous content and will not be repeated here.

[0137] Fusion of neighbor information: Aggregator 1 and Aggregator 2 are used to fuse neighbor information with their own node features, that is, to fuse the neighbor information and node feature information of the sample object to obtain the fused features of the sample object.

[0138] Training & Prediction: Concatenation refers to concatenating the sample object's own object features and fusion features to obtain the sample classification features, and then predicting through softmax processing to obtain the prediction results. Then, the model parameters can be updated based on the supervised training method, that is, the loss function value is calculated based on the difference between the prediction result and the node label, and then the model parameters are updated and retrained.

[0139] Step 460: Determine the target multimedia object associated with the designated multimedia object in the multimedia object network according to the classification result.

[0140] Specifically, the classification result of the multimedia object includes the probability that the multimedia object belongs to each object type. Finally, the object type corresponding to the maximum probability can be used as the final classification result of the multimedia object. For example, the output result is (0.8, 0.7, 0.6), then the object type corresponding to 0.8 can be used as the object type of the multimedia object.

[0141] In one embodiment of the present application, the classification model in this embodiment is a binary classification model, that is, there are two types of objects, such as abnormal multimedia objects and normal multimedia objects. It is assumed that the abnormal multimedia object is the target multimedia object that needs to be determined in the end. The classification result may include the probability that the multimedia object belongs to the abnormal multimedia object. When the probability is greater than a threshold, the multimedia object is considered to belong to the abnormal multimedia object. Exemplarily, the output result is 0.8. Assuming the threshold is 0.7, the multimedia object is an abnormal multimedia object, that is, the target multimedia object.

[0142] In the technical solution provided in the embodiment of the present application, a multimedia object network is constructed based on data extracted from object isomorphic graph data and object heterogeneous graph data, and then classification processing is performed according to the multimedia network to obtain a classification result, and then the target multimedia object is identified based on the classification result. The classification process fully considers the object feature data and neighbor information of the multimedia object itself. The classification model based on the graph convolutional neural network can start from the abnormality degree of the multimedia object itself, fully learn the characteristic abnormal nodes, and find the target multimedia object, and the detection accuracy is high.

[0143] Figure 6 The flowchart of a multimedia object detection method provided by an embodiment of the present application is schematically shown. This embodiment is a further refinement of the above embodiment. Figure 6 As shown, the multimedia object detection method provided in the embodiment of the present application includes steps 610 to 660, which are specifically as follows:

[0144] Step 610: Obtain object isomorphic graph data and object heterogeneous graph data corresponding to the specified multimedia object. The object isomorphic graph data is constructed based on the association relationship between multimedia objects. The object heterogeneous graph data is constructed based on the association relationship between multimedia objects and object diffusion media. The object diffusion media represents the dimension of similarity between multimedia objects.

[0145] Step 620: extract first multimedia object data associated with the designated multimedia object from the object isomorphism graph data.

[0146] Step 630: extract second multimedia object data associated with the designated multimedia object from the object heterogeneous graph data.

[0147] Step 640: Generate a multimedia object network using the multimedia objects respectively included in the first multimedia object data and the second multimedia object data as nodes.

[0148] For steps 610-640, reference may be made to the relevant descriptions of steps 210-240 or steps 410-440 in the aforementioned embodiment, which will not be repeated here.

[0149] Step 650: Calculate the similarity between the multimedia objects in the multimedia object network and the designated multimedia object to obtain the similarity result of the multimedia objects in the multimedia object network.

[0150] Specifically, the similarity between the multimedia object and the specified multimedia object can be determined based on similar items in the feature data of the two objects. For example, the consistency of 10 features in the feature data of the two objects is calculated. If 8 of them are consistent, the similarity can be set to 8 / 10*100%=80%.

[0151] In one embodiment of the present application, the similarity calculation process specifically includes: generating a second object relationship matrix based on the association relationships and relationship weights between multimedia objects in the multimedia object network; generating a similarity matrix between multimedia objects based on object feature data of the multimedia objects in the multimedia object network; generating an object node degree matrix based on the number of other multimedia objects directly connected to the multimedia objects in the multimedia object network; and calculating the similarity results of the multimedia objects in the multimedia object network based on the second object relationship matrix, the similarity matrix and the object node degree matrix.

[0152] Specifically, the relationship weight is the weight of the edge between nodes in the multimedia object network. The generation process of the second object relationship matrix is ​​similar to that of the first object relationship matrix. However, in the second object relationship matrix, when two multimedia objects have an association relationship, the corresponding elements in the matrix are not directly set to set values, but are set according to the relationship weight. Exemplarily, the second object relationship matrix includes multiple rows and multiple columns, and the elements corresponding to the i-th row and the j-th column represent the association relationship between the i-th node and the j-th node; if there is no association relationship between the two nodes, that is, there is no edge connecting the two nodes, then the elements corresponding to the two nodes in the second object relationship matrix are set to 0; if there is an association relationship between the two nodes, that is, the two nodes are directly connected by an edge, and the weight of the edge is 0.8, then the elements corresponding to the two nodes in the second object relationship matrix are set to 0.8.

[0153] The similarity matrix is ​​the similarity between multimedia objects calculated based on specific constraints. For example, Figure 3 The similarity included in the "object-object" data is used as an element in the similarity matrix. Of course, the similarity between multimedia objects can also be calculated based on other constraints. The object node degree matrix reflects the number of other multimedia objects directly connected to the multimedia object, that is, the number of other nodes directly connected to a node. For example, if a node is directly connected to three other nodes, then the node degree of the node is 3, and the element corresponding to the node in the object node degree matrix is ​​3.

[0154] Then, according to the second object relationship matrix, the similarity matrix and the object node degree matrix, the similarity results of the multimedia objects in the multimedia object network are calculated. This calculation process includes multiple iterative calculations. First, the second object relationship matrix is ​​multiplied by the similarity matrix to obtain a first matrix product, and the first matrix product is multiplied by the first weight to obtain a first calculation result; wherein the first weight is a preset value, and the first weight is in the range of 0 to 1. At the same time, the object node degree matrix is ​​multiplied by the object position matrix to obtain a second matrix product, and the second matrix product is multiplied by the second weight to obtain a second calculation result; wherein the object position matrix represents the position of the specified multimedia object in the multimedia object network. For example, assuming that there are 3 objects in total, the object position matrix is ​​a 3*3 matrix, and the specified multimedia object is the first one, then the elements corresponding to the first row and the first column in the object position matrix are set to 1, and the others are set to 0. The second weight is also a preset value, and the sum of the first weight and the second weight is 1, so the second weight can be obtained by subtracting the first weight from 1. In one embodiment, after the second matrix product is multiplied by the second weight, the product result is normalized to obtain the second calculation result.

[0155] Next, the first calculation result and the second calculation result are merged to obtain a candidate similarity result. For example, the first calculation result and the second calculation result are added to obtain a candidate similarity result.

[0156] If the difference between the candidate similarity result and the similarity result obtained by the previous calculation is greater than the preset threshold, the candidate similarity result is used as a new similarity matrix, and the candidate similarity result is recalculated based on the new similarity matrix. Of course, when the candidate similarity result is calculated for the first time, the candidate similarity result is directly used as a new similarity matrix and the iterative calculation is re-performed. If the difference between the candidate similarity result and the similarity result obtained by the previous calculation is less than or equal to the preset threshold, the candidate similarity result is used as the similarity result.

[0157] In one embodiment of the present application, the similarity results of multimedia objects in the multimedia object network are calculated based on the second object relationship matrix, the similarity matrix and the object node degree matrix, which can be implemented by a weighted walk algorithm. At this time, the first weight and the second weight involved in the aforementioned process are the hyperparameters of the walk algorithm, and the object position matrix represents the starting node when the algorithm is executed. When the candidate similarity results are obtained, if the difference between two or more adjacent candidate similarity results is not large, that is, the difference is less than the preset threshold, it can be considered that the algorithm converges to the equilibrium state of the Markov chain, the iteration stops, and the last candidate similarity result is used as the final similarity result. The calculation process of the weighted walk algorithm can consider the weight of the edge (represented by the second object relationship matrix), the degree of the node (represented by the object node degree matrix), the number of reachable paths of neighbors and seed nodes, and the distance of the path. Among them, the number of reachable paths and the distance are parameters involved in the calculation process of the walk algorithm. The reachable path refers to the path from one node to another node, and the number of reachable paths is the number of paths that a node can reach another node; the distance of the path refers to the number of nodes passed by the path. In some cases, the edge can also be used as a parameter of the distance of the path. Generally speaking, the larger the number of reachable paths between two nodes, the more similar the two nodes are. The closer the path distance between two nodes, the more similar the two nodes are.

[0158] For example, Figure 7A-7B The flowchart of a method for detecting multimedia objects based on similarity provided by an embodiment of the present application is schematically shown. Fig. 7A As shown, when performing multimedia object detection, first query in the graph database according to the black seed (i.e., the specified media object), the graph database includes object isomorphic graph data and object heterogeneous graph data, to obtain first multimedia object data and second multimedia object data, that is, the multi-order neighbors of the black seed are obtained by querying the graph database to form a multimedia object network, or a local network.

[0159] Next enter Figure 7B The steps shown in the figure generate a second object relationship matrix W, a similarity matrix H and an object node degree matrix D based on the multimedia object network, and generate an object position matrix Z according to the black seed position, and then according to the formula G = d*[W·H]+(1-d)([Z·D] norm ), where [] represents matrix calculation, norm represents normalization operation, d represents the first weight, (1-d) represents the second weight, and G represents the candidate similarity result. For example, Figure 7B The network shown in the figure consists of nodes A, B and C. The weight between nodes A and B is 1, the weight between nodes B and C is 0.5, and the weight between nodes A and C is 0.8. Based on these data, the following is generated: Figure 7B The weighted second object relationship matrix W is shown. In the example, the values ​​in the similarity matrix H are the same as the values ​​in the second object relationship matrix W. In actual situations, they may have different values. For the object node degree matrix D, it can be seen from the network that each node is connected to two other nodes, so the node degree corresponding to each node is 2. For the object position matrix Z, the black seed is node A, so the position of node A in the object position matrix Z is 1, and the others are 0. It can be understood that Figure 7B The unlabeled elements in each matrix are all 0. The above four matrices are fused and calculated to obtain the candidate similarity results, where the first weight is 0.85 and the second weight is 0.15.

[0160] Step 660: Determine the target multimedia object associated with the designated multimedia object in the multimedia object network according to the similarity result.

[0161] Specifically, the multimedia objects whose similarity in the similarity results is higher than a preset threshold may be used as target multimedia objects.

[0162] In the technical solution provided in the embodiment of the present application, a multimedia object network is constructed based on data extracted from object isomorphic graph data and object heterogeneous graph data, and then similarity calculation is performed based on the multimedia network, and then the target multimedia object is determined based on the similarity result. The similarity calculation process can fully consider parameters such as the relationship weights between multimedia objects, object feature data, and the number of other multimedia objects directly connected to the multimedia object, and can calculate accurate similarity results, thereby improving the detection accuracy of the target multimedia object.

[0163] In one embodiment of the present application, when detecting a target multimedia object, the target multimedia object can be determined by combining the classification result and the similarity result. Specifically, firstly, the multimedia objects in the multimedia object network are classified to obtain the classification result of the multimedia objects in the multimedia object network, and the first target multimedia object of the same type as the designated multimedia object in the multimedia object network is determined according to the classification result; at the same time, the similarity between the multimedia objects in the multimedia object network and the designated multimedia object is calculated to obtain the similarity result of the multimedia objects in the multimedia object network, and the second target multimedia object whose similarity with the designated multimedia object is greater than a preset threshold in the multimedia object network is determined according to the similarity result. Then, the target multimedia object associated with the designated multimedia object in the multimedia object network is generated according to the first target multimedia object and the second target multimedia object. For example, it is determined that there are 10 first target multimedia objects through classification processing, and 7 second target multimedia objects are determined through similarity calculation. If the first target multimedia object and the second target multimedia object do not overlap, it can be finally determined that there is a second target multimedia object.

[0164] By combining classification processing and similarity calculation to determine the final target multimedia object, the coverage and accuracy of multimedia object detection can be further improved, avoiding the situation where one of the processing methods misses the detection.

[0165] It should be noted that although the steps of the method in the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0166] The following introduces an apparatus embodiment of the present application, which can be used to execute the multimedia object detection method in the above-mentioned embodiment of the present application. Figure 8 The structure block diagram of the multimedia object detection device provided in the embodiment of the present application is schematically shown. Figure 8 As shown, the multimedia object detection device provided in the embodiment of the present application includes:

[0167] The data acquisition module 810 is used to acquire object isomorphic graph data and object heterogeneous graph data corresponding to a specified multimedia object, wherein the object isomorphic graph data is constructed based on the association relationship between multimedia objects, and the object heterogeneous graph data is constructed based on the association relationship between multimedia objects and object diffusion media, wherein the object diffusion media represents the dimension of similarity between multimedia objects;

[0168] A first data extraction module 820, configured to extract first multimedia object data associated with the specified multimedia object from the object isomorphism graph data;

[0169] A second data extraction module 830, configured to extract second multimedia object data associated with the specified multimedia object from the object heterogeneous graph data;

[0170] The object identification module 840 is configured to perform object identification according to the first multimedia object data and the second multimedia object data to determine a target multimedia object associated with the designated multimedia object.

[0171] In one embodiment of the present application, the object isomorphism graph data is constructed with multimedia objects as nodes and associations between multimedia objects as edges; the first data extraction module 820 is specifically used for:

[0172] Query the object isomorphism graph data according to the designated multimedia object, and take the designated multimedia object and a multimedia object whose number of nodes between the designated multimedia object and the designated multimedia object is less than or equal to a first designated number as a first multimedia object;

[0173] The object feature data of the first multimedia objects and the feature data of the relationships between the first multimedia objects are used as first multimedia object data.

[0174] In one embodiment of the present application, the object heterogeneous graph data is constructed with multimedia objects and object diffusion media as nodes and with associations between multimedia objects and object diffusion media as edges; the second data extraction module 830 is specifically used to:

[0175] Querying the object heterogeneous graph data according to the specified multimedia object to determine neighbor multimedia objects whose number of nodes separated from the specified multimedia object is less than or equal to a second specified number;

[0176] Generate relationship data between the specified multimedia object and the neighbor multimedia object according to the medium node data between the specified multimedia object and the neighbor multimedia object in the object isomorphism graph data; wherein the medium node data is used to represent the data of the node corresponding to the object diffusion medium;

[0177] The neighbor multimedia object and the designated multimedia object are used as second multimedia objects, and object feature data of the second multimedia objects and feature data of relationships between the second multimedia objects are used as second multimedia object data.

[0178] In one embodiment of the present application, the object identification module 840 includes:

[0179] A network construction unit, configured to generate a multimedia object network using the multimedia objects respectively included in the first multimedia object data and the second multimedia object data as nodes;

[0180] The object recognition unit is used to perform object recognition according to the multimedia object network to determine a target multimedia object associated with the designated multimedia object in the multimedia object network.

[0181] In one embodiment of the present application, the object recognition unit includes:

[0182] The classification processing subunit is used to classify the multimedia objects in the multimedia object network to obtain the classification results of the multimedia objects in the multimedia object network; and determine the target multimedia object associated with the specified multimedia object in the multimedia object network according to the classification results.

[0183] In one embodiment of the present application, the classification processing subunit is specifically used for:

[0184] Generate a first object relationship matrix according to the association relationship between the multimedia objects included in the multimedia object network;

[0185] generating an object feature matrix according to object feature data of multimedia objects contained in the multimedia object network;

[0186] Feature extraction and mapping processing are performed according to the first object relationship matrix and the object feature matrix to obtain classification results of multimedia objects in the multimedia object network.

[0187] In one embodiment of the present application, the classification processing subunit is specifically used for:

[0188] Extracting neighbor features of multimedia objects in the multimedia object network according to the first object relationship matrix, and extracting object feature data of multimedia objects in the multimedia object network according to the object feature matrix; the neighbor features of the multimedia objects include object feature data of other multimedia objects having an association relationship with the multimedia object;

[0189] fusing the neighbor information of the multimedia object and the node feature information to obtain a fusion feature of the multimedia object;

[0190] splicing the object feature data of the multimedia object with the fusion feature of the multimedia object to obtain the feature to be classified of the multimedia object;

[0191] Mapping processing is performed according to the to-be-classified features of the multimedia object to obtain a classification result of the multimedia object.

[0192] In one embodiment of the present application, the classification processing subunit is specifically used for:

[0193] Sampling multimedia objects in the multimedia object network to obtain a plurality of multimedia objects to be identified;

[0194] generating an object relationship matrix to be identified according to the association relationship between the plurality of multimedia objects to be identified in the multimedia object network, and generating an object feature matrix to be identified according to the object feature data of the plurality of multimedia objects to be identified in the multimedia object network;

[0195] Feature extraction and mapping processing are performed according to the to-be-identified object relationship matrix and the to-be-identified object feature matrix to obtain a classification result of the to-be-identified multimedia objects in the multimedia object network.

[0196] In one embodiment of the present application, the classification processing subunit is specifically used for:

[0197] According to the object diffusion medium type involved in establishing an association relationship between the multimedia objects in the multimedia object network and the designated multimedia object, and the sampling weight corresponding to the object diffusion medium type, the multimedia objects in the multimedia object network are sampled to obtain a plurality of multimedia objects to be identified.

[0198] In one embodiment of the present application, the object recognition unit includes:

[0199] The similarity recognition subunit is used to calculate the similarity between the multimedia objects in the multimedia object network and the specified multimedia object, and obtain the similarity results of the multimedia objects in the multimedia object network; and determine the target multimedia object associated with the specified multimedia object in the multimedia object network according to the similarity results.

[0200] In one embodiment of the present application, the similarity identification subunit is specifically used for:

[0201] Generate a second object relationship matrix according to the association relationships and relationship weights between multimedia objects in the multimedia object network;

[0202] generating a similarity matrix between multimedia objects according to object feature data of multimedia objects in the multimedia object network;

[0203] generating an object node degree matrix according to the number of other multimedia objects directly connected to the multimedia object in the multimedia object network;

[0204] The similarity results of multimedia objects in the multimedia object network are calculated according to the second object relationship matrix, the similarity matrix and the object node degree matrix.

[0205] In one embodiment of the present application, the similarity identification subunit is specifically used for:

[0206] Multiplying the second object relationship matrix by the similarity matrix to obtain a first matrix product, and multiplying the first matrix product by a first weight to obtain a first calculation result;

[0207] Multiplying the object node degree matrix and the object position matrix to obtain a second matrix product, and multiplying the second matrix product and a second weight to obtain a second calculation result; wherein the object position matrix represents the position of the specified multimedia object in the multimedia object network, and the sum of the first weight and the second weight is 1;

[0208] The first calculation result and the second calculation result are combined to obtain a candidate similarity result;

[0209] When the difference between the candidate similarity result and the similarity result calculated last time is greater than a preset threshold, taking the candidate similarity result as a new similarity matrix, and recalculating the candidate similarity result based on the new similarity matrix;

[0210] When the difference between the candidate similarity result and the similarity result obtained by the previous calculation is less than or equal to the preset threshold, the candidate similarity result is used as the similarity result.

[0211] In one embodiment of the present application, the object recognition unit is specifically used to:

[0212] Classifying the multimedia objects in the multimedia object network to obtain classification results of the multimedia objects in the multimedia object network;

[0213] Determine, according to the classification result, a first target multimedia object in the multimedia object network that is of the same type as the designated multimedia object;

[0214] Calculating the similarity between the multimedia objects in the multimedia object network and the specified multimedia object to obtain a similarity result of the multimedia objects in the multimedia object network;

[0215] Determine, according to the similarity result, a second target multimedia object in the multimedia object network whose similarity to the designated multimedia object is greater than a preset threshold;

[0216] A target multimedia object associated with the designated multimedia object in the multimedia object network is generated according to the first target multimedia object and the second target multimedia object.

[0217] The specific details of the multimedia object detection device provided in each embodiment of the present application have been described in detail in the corresponding method embodiment, and will not be repeated here.

[0218] Fig. 9 The structure block diagram of a computer system for implementing an electronic device according to an embodiment of the present application is schematically shown.

[0219] It should be noted that Fig. 9 The computer system 900 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0220] like Fig. 9As shown, the computer system 900 includes a central processing unit 901 (CPU), which can perform various appropriate actions and processes according to the program stored in the read-only memory 902 (ROM) or the program loaded from the storage part 908 to the random access memory 903 (RAM). Various programs and data required for system operation are also stored in the random access memory 903. The central processing unit 901, the read-only memory 902 and the random access memory 903 are connected to each other through a bus 904. An input / output interface 905 (Input / Output interface, i.e., I / O interface) is also connected to the bus 904.

[0221] The following components are connected to the input / output interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed so that a computer program read therefrom is installed into the storage section 908 as needed.

[0222] In particular, according to an embodiment of the present application, the process described in each method flow chart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program contains a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the central processor 901, various functions defined in the system of the present application are executed.

[0223] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0224] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0225] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0226] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0227] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary technical means in the art that are not disclosed in the present application.

[0228] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for detecting multimedia objects, It is characterized in that include: Obtaining object isomorphic graph data and object heterogeneous graph data corresponding to a specified multimedia object, wherein the object isomorphic graph data is constructed based on associations between multimedia objects, and the object heterogeneous graph data is constructed based on associations between multimedia objects and object diffusion media, wherein the object diffusion media represents dimensions of similarity between multimedia objects; Extracting first multimedia object data having an association relationship with the specified multimedia object from the object isomorphism graph data; Extracting second multimedia object data having an association relationship with the specified multimedia object from the object heterogeneous graph data; Object recognition is performed according to the first multimedia object data and the second multimedia object data to determine a target multimedia object associated with the designated multimedia object.

2. The method for detecting multimedia objects according to claim 1, It is characterized in that The object isomorphism graph data is constructed with multimedia objects as nodes and association relationships between multimedia objects as edges; extracting first multimedia object data having an association relationship with the specified multimedia object from the object isomorphism graph data includes: Query the object isomorphism graph data according to the designated multimedia object, and take the designated multimedia object and a multimedia object whose number of nodes between the designated multimedia object and the designated multimedia object is less than or equal to a first designated number as a first multimedia object; The object feature data of the first multimedia objects and the feature data of the relationships between the first multimedia objects are used as first multimedia object data.

3. The method for detecting multimedia objects according to claim 1, It is characterized in that The object heterogeneous graph data is constructed with multimedia objects and object diffusion media as nodes and with associations between multimedia objects and object diffusion media as edges; extracting second multimedia object data having an association relationship with the specified multimedia object from the object heterogeneous graph data includes: Querying the object heterogeneous graph data according to the specified multimedia object to determine neighbor multimedia objects whose number of nodes separated from the specified multimedia object is less than or equal to a second specified number; Generate relationship data between the specified multimedia object and the neighbor multimedia object according to the medium node data between the specified multimedia object and the neighbor multimedia object in the object isomorphism graph data; wherein the medium node data is used to represent the data of the node corresponding to the object diffusion medium; The neighbor multimedia object and the designated multimedia object are used as second multimedia objects, and object feature data of the second multimedia objects and feature data of relationships between the second multimedia objects are used as second multimedia object data.

4. The method for detecting a multimedia object according to claim 1, It is characterized in that Performing object recognition according to the first multimedia object data and the second multimedia object data to determine a target multimedia object associated with the designated multimedia object includes: Generate a multimedia object network using the multimedia objects respectively included in the first multimedia object data and the second multimedia object data as nodes; Object recognition is performed according to the multimedia object network to determine a target multimedia object associated with the designated multimedia object in the multimedia object network.

5. The method for detecting multimedia objects according to claim 4, It is characterized in that Performing object recognition according to the multimedia object network to determine a target multimedia object associated with the designated multimedia object in the multimedia object network includes: Classifying the multimedia objects in the multimedia object network to obtain classification results of the multimedia objects in the multimedia object network; A target multimedia object associated with the designated multimedia object in the multimedia object network is determined according to the classification result.

6. The method for detecting a multimedia object according to claim 5, It is characterized in that Classifying the multimedia objects in the multimedia object network to obtain classification results of the multimedia objects in the multimedia object network includes: Generate a first object relationship matrix according to the association relationship between the multimedia objects included in the multimedia object network; generating an object feature matrix according to object feature data of multimedia objects contained in the multimedia object network; Feature extraction and mapping processing are performed according to the first object relationship matrix and the object feature matrix to obtain classification results of multimedia objects in the multimedia object network.

7. The method for detecting a multimedia object according to claim 6, It is characterized in that Performing feature extraction and mapping processing according to the first object relationship matrix and the object feature matrix to obtain classification results of multimedia objects in the multimedia object network includes: Extracting neighbor features of multimedia objects in the multimedia object network according to the first object relationship matrix, and extracting object feature data of multimedia objects in the multimedia object network according to the object feature matrix; the neighbor features of the multimedia objects include object feature data of other multimedia objects having an association relationship with the multimedia object; fusing the neighbor information of the multimedia object and the node feature information to obtain a fusion feature of the multimedia object; splicing the object feature data of the multimedia object with the fusion feature of the multimedia object to obtain the feature to be classified of the multimedia object; Mapping processing is performed according to the to-be-classified features of the multimedia object to obtain a classification result of the multimedia object.

8. The method for detecting multimedia objects according to claim 5, It is characterized in that Classifying the multimedia objects in the multimedia object network to obtain classification results of the multimedia objects in the multimedia object network includes: Sampling multimedia objects in the multimedia object network to obtain a plurality of multimedia objects to be identified; generating an object relationship matrix to be identified according to the association relationship between the plurality of multimedia objects to be identified in the multimedia object network, and generating an object feature matrix to be identified according to the object feature data of the plurality of multimedia objects to be identified in the multimedia object network; Feature extraction and mapping processing are performed according to the to-be-identified object relationship matrix and the to-be-identified object feature matrix to obtain a classification result of the to-be-identified multimedia objects in the multimedia object network.

9. The method for detecting multimedia objects according to claim 8, It is characterized in that Sampling multimedia objects in the multimedia object network to obtain a plurality of multimedia objects to be identified includes: According to the object diffusion medium type involved in establishing an association relationship between the multimedia objects in the multimedia object network and the designated multimedia object, and the sampling weight corresponding to the object diffusion medium type, the multimedia objects in the multimedia object network are sampled to obtain a plurality of multimedia objects to be identified.

10. The method for detecting multimedia objects according to claim 4, It is characterized in that Performing object recognition according to the multimedia object network to determine a target multimedia object associated with the designated multimedia object in the multimedia object network includes: Calculating the similarity between the multimedia objects in the multimedia object network and the specified multimedia object to obtain a similarity result of the multimedia objects in the multimedia object network; A target multimedia object associated with the designated multimedia object in the multimedia object network is determined according to the similarity result.

11. The method for detecting multimedia objects according to claim 10, It is characterized in that Calculating the similarity between the multimedia objects in the multimedia object network and the designated multimedia object to obtain the similarity result of the multimedia objects in the multimedia object network includes: Generate a second object relationship matrix according to the association relationships and relationship weights between multimedia objects in the multimedia object network; generating a similarity matrix between multimedia objects according to object feature data of multimedia objects in the multimedia object network; generating an object node degree matrix according to the number of other multimedia objects directly connected to the multimedia object in the multimedia object network; The similarity results of multimedia objects in the multimedia object network are calculated according to the second object relationship matrix, the similarity matrix and the object node degree matrix.

12. The method for detecting multimedia objects according to claim 11, It is characterized in that Calculating similarity results of multimedia objects in the multimedia object network according to the second object relationship matrix, the similarity matrix and the object node degree matrix includes: Multiplying the second object relationship matrix by the similarity matrix to obtain a first matrix product, and multiplying the first matrix product by a first weight to obtain a first calculation result; Multiplying the object node degree matrix and the object position matrix to obtain a second matrix product, and multiplying the second matrix product and a second weight to obtain a second calculation result; wherein the object position matrix represents the position of the specified multimedia object in the multimedia object network, and the sum of the first weight and the second weight is 1; The first calculation result and the second calculation result are combined to obtain a candidate similarity result; When the difference between the candidate similarity result and the similarity result calculated last time is greater than a preset threshold, taking the candidate similarity result as a new similarity matrix, and recalculating the candidate similarity result based on the new similarity matrix; When the difference between the candidate similarity result and the similarity result obtained by the previous calculation is less than or equal to the preset threshold, the candidate similarity result is used as the similarity result.

13. The method for detecting multimedia objects according to claim 4, It is characterized in that Performing object recognition according to the multimedia object network to determine a target multimedia object associated with the designated multimedia object in the multimedia object network includes: Classifying the multimedia objects in the multimedia object network to obtain classification results of the multimedia objects in the multimedia object network; Determine, according to the classification result, a first target multimedia object in the multimedia object network that is of the same type as the designated multimedia object; Calculating the similarity between the multimedia objects in the multimedia object network and the specified multimedia object to obtain a similarity result of the multimedia objects in the multimedia object network; Determine, according to the similarity result, a second target multimedia object in the multimedia object network whose similarity to the designated multimedia object is greater than a preset threshold; A target multimedia object associated with the designated multimedia object in the multimedia object network is generated according to the first target multimedia object and the second target multimedia object.

14. A detection device for a multimedia object, It is characterized in that include: A data acquisition module, used to acquire object isomorphic graph data and object heterogeneous graph data corresponding to a specified multimedia object, wherein the object isomorphic graph data is constructed based on the association relationship between multimedia objects, and the object heterogeneous graph data is constructed based on the association relationship between multimedia objects and object diffusion media, wherein the object diffusion media represents the dimension of similarity between multimedia objects; A first data extraction module, configured to extract first multimedia object data associated with the specified multimedia object from the object isomorphism graph data; A second data extraction module, configured to extract second multimedia object data associated with the specified multimedia object from the object heterogeneous graph data; The object recognition module is used to perform object recognition according to the first multimedia object data and the second multimedia object data to determine a target multimedia object associated with the designated multimedia object.

15. A computer readable medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the method for detecting a multimedia object according to any one of claims 1 to 13 is implemented.

16. An electronic device, It is characterized in that include: processor; as well as A memory, configured to store executable instructions of the processor; The processor executes the executable instructions so that the electronic device executes the multimedia object detection method described in any one of claims 1 to 13.

17. A computer program product, It is characterized in that The computer program product includes computer instructions stored in a computer-readable storage medium; The processor of the computer device reads and executes the computer instructions from the computer-readable storage medium, so that the computer device executes the multimedia object detection method according to any one of claims 1 to 13.

Images