Object relationship mining method and device and storage medium

Abstract

The invention discloses an object relation mining method and device and a storage medium, and the method comprises the steps: building an object relation graph based on the perception data of each to-be-analyzed object, enabling the nodes in the object relation graph to represent the to-be-analyzed objects, and enabling the edges in the object relation graph to represent the incidence relation between the to-be-analyzed objects; converting information corresponding to nodes and / or edges in the object relation graph into a natural language to obtain a description text corresponding to the object relation graph; and inputting a description text corresponding to the object relation graph into the trained relation mining model to obtain an object relation mining result output by the relation mining model. The structural information contained in the object relation graph is converted into the natural language, the deep semantic information of the perception data is fully mined, the structural information of the object relation graph and the natural language of the deep semantic information are combined, and the accuracy of the object relation mining result obtained through final mining is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an object relationship mining method, device, and storage medium. Background Art

[0002] At present, based on the early construction of the perception device network, a large amount of perception data has been accumulated, such as image acquisition data of the objects to be analyzed and device acquisition data of terminal devices.

[0003] Perception data contains a wealth of hidden information. Existing methods can mine and analyze the group structure of the subjects being analyzed based on perception data, understanding the relationships between groups and the dynamics within them. However, the analysis of perception data remains incomplete and inaccurate, hindering its effective mining and application.

[0004] Therefore, how to conduct accurate and comprehensive group structure mining and analysis of perception data to maximize the value of these perception data is an urgent problem to be solved by those skilled in the art. Summary of the Invention

[0005] In order to solve the above technical problems, the present application at least provides an object relationship mining method, device and storage medium.

[0006] In a first aspect, the present application provides an object relationship mining method, which includes: obtaining attribute data and / or trajectory data corresponding to each object to be analyzed to obtain perception data of each object to be analyzed; constructing an object relationship graph based on the perception data of each object to be analyzed, wherein the nodes in the object relationship graph represent the objects to be analyzed, and the edges in the object relationship graph represent the association relationships between the objects to be analyzed; converting the information corresponding to the nodes and / or edges in the object relationship graph into natural language to obtain a descriptive text corresponding to the object relationship graph; inputting the descriptive text corresponding to the object relationship graph into a trained relationship mining model to obtain an object relationship mining result output by the relationship mining model.

[0007] In one embodiment, information corresponding to nodes and / or edges in an object relationship graph is converted into natural language to obtain descriptive text corresponding to the object relationship graph, including: performing group relationship mining on the object relationship graph to divide the object relationship graph into multiple group subgraphs; and converting information corresponding to nodes and / or edges in each group subgraph into natural language to obtain descriptive text corresponding to each group subgraph.

[0008] In one embodiment, the information corresponding to the nodes and / or edges in each group sub-graph is respectively converted into natural language to obtain the description text corresponding to each group sub-graph, including: converting the attribute information corresponding to each node and / or each edge in the group sub-graph into natural language statements; concatenating the natural language statements corresponding to each attribute information to respectively obtain the node description text corresponding to each node and / or the edge description text corresponding to each edge; and performing summary analysis on the node description text corresponding to each node and / or the edge description text corresponding to each edge to obtain the summary description text corresponding to the group sub-graph.

[0009] In one embodiment, the description text corresponding to the object relationship graph is input into the trained relationship mining model to obtain the object relationship mining result output by the relationship mining model, including: inputting the description text corresponding to the group sub-graph into the trained relationship mining model, taking the group type corresponding to the group sub-graph output by the relationship mining model as the target group type, and taking the association relationship between each object to be analyzed in the group sub-graph output by the relationship mining model as the target object relationship.

[0010] In one embodiment, the method further includes: obtaining the preset available association relationships corresponding to the target group type to obtain the set of available relationships; detecting whether each target object relationship corresponding to the group sub-graph is in the set of available relationships; marking the edges corresponding to the target object relationships that are not in the set of available relationships as the edges with analysis anomalies, and marking the group sub-graph as the group sub-graph with analysis anomalies.

[0011] In one embodiment, the group relationship mining also obtains the group type corresponding to the group sub-graph, taking the group type obtained by the group relationship mining as the initial group type, and taking the association relationship between each object to be analyzed represented by the original edges in the group sub-graph as the initial object relationship; the method further includes: detecting the matching degree between the target group type and the initial group type of the group sub-graph, and / or detecting the matching degree between the target object relationship and the initial object relationship between each object to be analyzed in the group sub-graph; marking the group sub-graph with a matching degree lower than the threshold between the target group type and the initial group type as the group sub-graph with analysis anomalies, and / or marking the edges with a matching degree lower than the threshold between the target object relationship and the initial object relationship as the edges with analysis anomalies.

[0012] In one embodiment, the method further includes: taking the group sub-graph marked with analysis anomalies and the edges marked with analysis anomalies as the abnormal graph information; displaying the abnormal graph information on the analysis result display page; receiving the judgment result input for the abnormal graph information on the analysis result display page, where the judgment result is used to represent whether the abnormal graph information has anomalies; and modifying the mark corresponding to the abnormal graph information based on the judgment result.

[0013] In one embodiment, the relationship mining model stores natural language definitions corresponding to multiple candidate group types and multiple candidate object relationships respectively; input the description text corresponding to the group subgraph into the trained relationship mining model, the relationship mining model outputs the group type corresponding to the group subgraph to obtain the target group type, and the relationship mining model outputs the association relationships between each object to be analyzed in the group subgraph to obtain the target object relationship, including: calculating the matching degree between the description text corresponding to the object relationship graph and the natural language definition corresponding to each candidate group type by using the relationship mining model, and selecting the candidate group type whose matching degree meets the preset condition as the target group type; and calculating the matching degree between the description text corresponding to the object relationship graph and the natural language definition corresponding to each candidate object relationship by using the relationship mining model, and selecting the candidate object relationship whose matching degree meets the preset condition as the target object relationship.

[0014] The second aspect of the present application provides an object relationship mining device, including: a data acquisition module, configured to acquire attribute data and / or trajectory data corresponding to each object to be analyzed to obtain the perception data of each object to be analyzed; a graph construction module, configured to construct an object relationship graph based on the perception data of each object to be analyzed, where the nodes in the object relationship graph represent the objects to be analyzed, and the edges in the object relationship graph represent the association relationships between the objects to be analyzed; a language conversion module, configured to convert the information corresponding to the nodes and / or edges in the object relationship graph into natural language to obtain the description text corresponding to the object relationship graph; a relationship mining module, configured to input the description text corresponding to the object relationship graph into the trained relationship mining model to obtain the object relationship mining result output by the relationship mining model.

[0015] The third aspect of the present application provides an electronic device, including a memory and a processor, and the processor is configured to execute program instructions stored in the memory to implement the above object relationship mining method.

[0016] The fourth aspect of the present application provides a computer-readable storage medium, on which program instructions are stored, and when the program instructions are executed by a processor, the above object relationship mining method is implemented.

[0017] In the above solution, perception data of each object to be analyzed is obtained by acquiring attribute data and / or trajectory data corresponding to each object to be analyzed; an object relationship graph is constructed based on the perception data of each object to be analyzed, where nodes in the object relationship graph represent objects to be analyzed, and edges in the object relationship graph represent the association relationships between objects to be analyzed; information corresponding to nodes and / or edges in the object relationship graph is converted into natural language to obtain a description text corresponding to the object relationship graph; the description text corresponding to the object relationship graph is input into a trained relationship mining model to obtain an object relationship mining result output by the relationship mining model, so as to convert the structured information contained in the object relationship graph into natural language, fully mine the deep semantic information of the perception data, combine the structured information of the object relationship graph and the natural language of the deep semantic information, and improve the accuracy of the finally mined object relationship mining result.

[0018] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit this application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings here are incorporated into the specification and form a part of this specification. These drawings show embodiments consistent with this application and are used together with the specification to explain the technical solutions of this application.

[0020] Figure 1 It is a schematic diagram of the solution implementation environment shown in an exemplary embodiment of this application;

[0021] Figure 2 It is a flowchart of an object relationship mining method shown in an exemplary embodiment of this application;

[0022] Figure 3 It is a schematic diagram of dividing an object relationship graph shown in an exemplary embodiment of this application;

[0023] Figure 4 It is a schematic diagram of an analysis result display page shown in an exemplary embodiment of this application;

[0024] Figure 5 It is a block diagram of an object relationship mining device shown in an exemplary embodiment of this application;

[0025] Figure 6 It is a schematic diagram of the structure of an electronic device shown in an exemplary embodiment of this application;

[0026] Figure 7 It is a schematic diagram of the structure of a computer-readable storage medium shown in an exemplary embodiment of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] The solutions of the embodiments of this application will be described in detail below with reference to the accompanying drawings of the specification.

[0028] In the following description, specific details such as specific system architectures, interfaces, technologies, etc. are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the present application.

[0029] The term “and / or” in this article is merely an associative information describing associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character “ / ” in this article generally represents an “or” relationship between the associated objects before and after. In addition, “multiple” in this article means two or more than two. In addition, the term “at least one kind” in this article represents any one of multiple kinds or any combination of at least two of multiple kinds. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set composed of A, B, and C.

[0030] The object relationship mining method provided by the embodiments of the present application will be described below.

[0031] Please refer to Figure 1 , Figure 1 is a schematic diagram of the solution implementation environment shown in an exemplary embodiment of the present application. The solution implementation environment may include a terminal 110 and a server 120, and the terminal 110 and the server 120 are communicatively connected to each other.

[0032] The number of terminals 110 can be one or more. The terminal 110 can be a camera, a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto.

[0033] The server 120 can be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.

[0034] In one example, the server 120 can perform object relationship mining on the perception data of each object to be analyzed obtained from the terminal 110, and obtain an object relationship mining result. Of course, the server 120 can store the object relationship mining result locally, send it back to the terminal 110, or transmit it to other terminals.

[0035] In one example, a client of a target application is installed and running in the terminal 110. For example, the target application can be an application that provides object relationship mining functions. The perception data of each object to be analyzed is mined for object relationships using the target application to obtain object relationship mining results. The server 120 can be the background server of the target application, used to provide background services for the client of the target application.

[0036] In the object relationship mining method provided by the embodiments of the present application, the execution entity of each step can be the terminal 110, such as the client of the target application installed and running in the terminal 110, or the server 120, or executed by the interaction and cooperation of the terminal 110 and the server 120, that is, a part of the steps of the method are executed by the terminal 110 and another part of the steps are executed by the server 120.

[0037] Please refer to Figure 2 , Figure 2 which is a flowchart of the object relationship mining method shown in an exemplary embodiment of the present application. The object relationship mining method can be applied to Figure 1 the shown implementation environment and is specifically executed by the server in this implementation environment. It should be understood that this method can also be applicable to other exemplary implementation environments and be specifically executed by devices in other implementation environments. This embodiment does not limit the implementation environment applicable to this method.

[0038] As Figure 2 shown, the object relationship mining method at least includes steps S210 to S240, which are introduced in detail as follows:

[0039] Step S210: Obtain the attribute data and / or trajectory data corresponding to each object to be analyzed to obtain the perception data of each object to be analyzed.

[0040] Among them, the objects to be analyzed include but are not limited to vehicles, ships, aircraft, animals, locations, or buildings, etc.

[0041] The attribute data of the object to be analyzed includes but is not limited to data such as the identifier, image, or model of the object to be analyzed; the trajectory data of the object to be analyzed includes but is not limited to the locations, regions, times, or residence durations passed by the object to be analyzed.

[0042] Obtain the attribute data and / or trajectory data corresponding to each object to be analyzed, and use these data as the perception data of each object to be analyzed.

[0043] Optionally, the obtained attribute data and / or trajectory data corresponding to the object to be analyzed can be subjected to data cleaning, and the attribute data and / or trajectory data after data cleaning are used as the perception data of each object to be analyzed.

[0044] Step S220: Construct an object relationship graph based on the perception data of each object to be analyzed. The nodes in the object relationship graph represent the objects to be analyzed, and the edges in the object relationship graph represent the association relationships between the objects to be analyzed.

[0045] Take each object to be analyzed as a node, and analyze the association relationships between the objects to be analyzed based on the perception data of each object to be analyzed, so as to construct edges between the nodes of the objects to be analyzed.

[0046] Among them, the specific type of the association relationship can be flexibly set according to different application scenarios. For example, there is a parking and custody relationship between a vehicle and a parking lot, and there is an accompanying driving relationship between vehicles.

[0047] Exemplarily, an object relationship graph can be constructed based on existing relationship mining algorithms. For example, define nodes and edges, and then construct an object relationship graph based on a network analysis library (such as NetworkX).

[0048] Exemplarily, the perception data of each object to be analyzed can also be input into a pre-trained relationship graph construction model to obtain the object relationship graph output by the relationship graph construction model.

[0049] Exemplarily, first evaluate the association degree between the objects to be analyzed according to an association degree evaluation model. The association degree evaluation model evaluates the association degree between the objects to be analyzed according to the association influence features in the perception data. The value of the final association degree can be between 0 and 1. The larger the value, the closer the relationship between the two objects to be analyzed. Among them, the association influence features include but are not limited to trajectory coincidence features, image features, etc.

[0050] Optionally, the association degree can be compared with a preset threshold. If the association degree between the objects to be analyzed is greater than the preset threshold, it is determined that there is an association relationship between them; otherwise, it is determined that there is no association relationship.

[0051] After quantifying the association degree between the objects to be analyzed, then determine the relationship type between the objects to be analyzed. Combine the association degree and the relationship type to obtain the association relationship between the objects to be analyzed, so as to construct an object relationship graph.

[0052] For example, for the objects to be analyzed determined to have an association relationship, the relationship classification can be further refined. For example, input the extracted relationship influence features into a relationship recognition model. The relationship recognition model has a relationship recognition function to obtain the specific relationship type between the objects to be analyzed. Among them, the above-mentioned association degree evaluation model and relationship recognition model can adopt machine learning models, which are not limited in this application.

[0053] Step S230: Convert the information corresponding to the nodes and / or edges in the object relationship graph into natural language to obtain the descriptive text corresponding to the object relationship graph.

[0054] Convert the structured information corresponding to the object relationship graph into natural language to obtain the descriptive text corresponding to the object relationship graph.

[0055] It can be to convert the information corresponding to each node and / or edge in the object relationship graph into natural language.

[0056] For example, the attribute information of the node corresponding to the vehicle in the object relationship graph is: {"id": "0001", "type": "sedan", "color": "black", "parking lot": "Shopping Mall A"}. Converting this node into natural language gives: This black sedan has a license plate number of 0001 and is parked and stored in Shopping Mall A.

[0057] Exemplarily, the descriptive text corresponding to the object relationship graph can be directly obtained by combining the natural language converted from each node and / or edge. For example, save the natural language converted from the nodes and / or edges to the corresponding nodes and / or edges of the object relationship graph to obtain the descriptive text corresponding to the object relationship graph; it is also possible to combine the natural language converted from each node and / or edge to comprehensively generate a summary descriptive text of the object relationship graph, and use the summary descriptive text as the descriptive text corresponding to the object relationship graph; it is also possible to save the natural language converted from the nodes and / or edges to the corresponding nodes and / or edges of the object relationship graph, and comprehensively generate a summary descriptive text of the object relationship graph, and use both the natural language of the nodes and / or edges and the summary descriptive text as the descriptive text corresponding to the object relationship graph.

[0058] Step S240: Input the descriptive text corresponding to the object relationship graph into the trained relationship mining model to obtain the object relationship mining result output by the relationship mining model.

[0059] Input the descriptive text corresponding to the object relationship graph into the trained relationship mining model so that the relationship mining model performs text feature analysis on the input descriptive text to obtain the object relationship mining results of each object to be analyzed finally.

[0060] Of course, in addition to only inputting the descriptive text corresponding to the object relationship graph into the relationship mining model for analysis, it is also possible to input both the object relationship graph and the descriptive text corresponding to the object relationship graph into the relationship mining model for analysis to obtain a more accurate object relationship mining result.

[0061] Among them, the object relationship mining result of the object to be analyzed can include the association relationship between two objects to be analyzed, and can also include the group relationship between multiple objects to be analyzed, which can be specifically determined according to the requirements of the application scenario, and this application does not limit this.

[0062] It should be noted that the relationship mining model can be implemented based on network architectures such as Convolutional Neural Networks (CNN), Recurrent Neural Network (RNN), or Large Language Model (LLM). The present application does not limit the specific architecture of the relationship mining model. The training process of the relationship mining model is similar to the above data processing process. The difference is that the model training loss needs to be calculated based on the difference between the true value label and the actual output, and the model parameters are adjusted according to the model training loss until the loss converges, obtaining a trained relationship mining model.

[0063] Compared with directly performing relationship mining on perception data in the prior art, the present application first uses perception data for initial relationship mining to construct an object relationship graph, and then converts the structured information contained in the object relationship graph into natural language, fully mining the deep semantic information of the perception data, so as to combine the structured information of the object relationship graph and the natural language of the deep semantic information, improving the accuracy of the object relationship mining result finally obtained.

[0064] Next, some embodiments of the present application will be described in detail.

[0065] In some embodiments, in step S230, converting the information corresponding to the nodes and / or edges in the object relationship graph into natural language to obtain a description text corresponding to the object relationship graph includes:

[0066] Step S231: Perform group relationship mining on the object relationship graph to divide the object relationship graph into multiple group subgraphs.

[0067] Among them, existing group relationship mining algorithms (such as Louvain algorithm, Girvan - Newman algorithm, etc.) can be used to mine the community groups existing in the object relationship graph. A community group refers to a group of nodes with a large degree of relevance in the network, thus forming a group structure with a tight internal connection and sparse external connection.

[0068] Exemplarily, please refer to Figure 3 , Figure 3 is a schematic diagram showing the division of the object relationship graph shown in an exemplary embodiment of the present application. As Figure 3 shown, performing group relationship mining on the object relationship graph obtains multiple social groups in the object relationship graph, and dividing the object relationship graph with community groups as the basic unit obtains multiple group subgraphs.

[0069] Optionally, the upper limit of the number of nodes contained in the group subgraph can be defined. For example, if the upper limit of the number of nodes is 10, then the graph with the number of nodes exceeding the upper limit is divided to avoid a super-large graph caused by a super node having connection relationships with numerous nodes.

[0070] Step S232: Convert the information corresponding to the nodes and / or edges in each group subgraph into natural language respectively, and obtain the description text corresponding to each group subgraph.

[0071] By converting the information corresponding to the nodes and / or edges in the group subgraph into natural language, it is convenient to mine and obtain more accurate deep semantic information.

[0072] Exemplarily, convert the attribute information corresponding to each node and / or each edge in the group subgraph into natural language statements; splice the natural language statements corresponding to each attribute information to obtain the node description text corresponding to each node and / or the edge description text corresponding to each edge respectively; perform summary analysis on the node description text corresponding to each node and / or the edge description text corresponding to each edge to obtain the summary description text corresponding to the group subgraph.

[0073] Illustrate with an example. Add the attribute information of the node and / or edge to the preset conversion template, and use the filled preset conversion template as the input of the pre-trained natural language conversion model to obtain the description text corresponding to the object relationship graph output by the natural language conversion model.

[0074] For example, the preset conversion template can be expressed as:

[0075] "You are a Chinese language expert, focusing on researching various language expression methods. You have extremely strong semantic understanding ability, can convert structured information into natural language, and keep the meaning unchanged.

[0076] You must act according to the following instructions:

[0077] 1. Perform natural language conversion according to the structured information;

[0078] 2. Only include the information in the structured data in the output, and do not fabricate randomly;

[0079] Input: {input_info}."

[0080] Where input_info is the attribute information corresponding to the node and / or edge.

[0081] Taking the natural language conversion of the node as an example, the node contains multiple attribute information. Based on each attribute information, fill it into the above preset conversion template, and use the filled preset conversion template as the input of the pre-trained natural language conversion model to obtain the natural language corresponding to this node output by the natural language conversion model.

[0082] When converting attribute information into natural language statements, multiple attribute information can be fused and converted into one natural language statement. For example, multiple attribute information of a vehicle node {"id": "0001", "type": "sedan", "color": "black"} is fused and converted into a natural language statement that the license plate number of this black sedan is 0001; it can also be that one attribute information is converted into one natural statement. For example, the above vehicle also contains attribute information {"parking lot": "Mall A"}, and converting this attribute information results in a natural language statement that the vehicle is parked in Mall A.

[0083] According to the natural language logical relationship between natural language statements, concatenate to obtain the node description text corresponding to each node and / or the edge description text corresponding to each edge.

[0084] Then, summarize and analyze the node description text corresponding to each node and / or the edge description text corresponding to each edge to obtain the summary description text corresponding to the group subgraph. This summary description text is a summary and refinement of the information in the group subgraph. For example, the number of vehicles, colors, models, parking location distributions included in the group subgraph, as well as the degree of association or frequency of association between vehicles, and whether there are abnormal trajectories, etc., are natural languages with summary significance.

[0085] For example, input the node description text corresponding to each node and / or the edge description text corresponding to each edge into a pre-trained natural language summary model to obtain the summary description text corresponding to the group subgraph output by the natural language summary model.

[0086] Illustrate with an example. Fill the node description text and / or the edge description text into a preset summary template, and use the filled preset summary template as the input of the natural language summary model to obtain the summary description text corresponding to the group subgraph output by the natural language summary model.

[0087] For example, the preset summary template can be expressed as:

[0088] "You are an expert in the Chinese language, focusing on researching various language expression methods. You have super semantic understanding ability and can summarize and extract a large amount of information.

[0089] You must follow the following instructions:

[0090] 1. Summarize according to the given large amount of information;

[0091] 2. Only include the given information in the output, and do not fabricate;

[0092] 3. The core information of the given data cannot be lost in the summary result;

[0093] >>>....>>>

[0094] The given information is between >>>. Please summarize and refine according to the given information.

[0095] Fill the node description text and / or edge description text into the >>> of the preset summary template to obtain the filled preset summary template.

[0096] Then, according to the summary description text corresponding to the obtained group sub-graph, perform relationship mining on the group sub-graph.

[0097] In some embodiments, input the description text corresponding to the object relationship graph into the trained relationship mining model, and obtain the object relationship mining result output by the relationship mining model, including: input the description text corresponding to the group sub-graph into the trained relationship mining model, take the group type corresponding to the group sub-graph output by the relationship mining model as the target group type, and take the association relationship between each object to be analyzed in the group sub-graph output by the relationship mining model as the target object relationship.

[0098] That is, relationship mining can include mining of group relationships and / or mining of association relationships between objects to be analyzed.

[0099] The relationship mining model includes a group relationship mining network and an object relationship mining network. It can be to input the description text corresponding to the group sub-graph into the group relationship mining network and the object relationship mining network respectively, obtain the target group type corresponding to the group sub-graph output by the group relationship mining network, and obtain the target object relationship between each object to be analyzed in the group sub-graph output by the object relationship mining network, where the target object relationship includes a relationship type and an association degree.

[0100] For example, the relationship mining model stores natural language definitions corresponding to multiple candidate group types and multiple candidate object relationships respectively; use the relationship mining model to calculate the matching degree between the description text corresponding to the object relationship graph and the natural language definition corresponding to each candidate group type, and select the candidate group type whose matching degree meets the preset conditions as the target group type; and use the relationship mining model to calculate the matching degree between the description text corresponding to the object relationship graph and the natural language definition corresponding to each candidate object relationship, and select the candidate object relationship whose matching degree meets the preset conditions as the target object relationship.

[0101] It can be to select the candidate group type or candidate object relationship with the largest matching degree, or it can also be to select the candidate group type or candidate object relationship with the largest matching degree and greater than the preset matching degree threshold. This application does not limit this.

[0102] Optionally, if the match with the candidate group type or candidate object relationship is successful, save the result. If the match fails, reuse the relationship mining model for relationship mining until the match is successful.

[0103] Optionally, for the mining of the association relationships between the objects to be analyzed, only the information corresponding to the edges in the object relationship graph can be converted into natural language to obtain edge description text, and the edge description text is input into the relationship mining model for relationship mining.

[0104] In some embodiments, the method further includes: obtaining the preset available association relationships corresponding to the target group type to obtain an available relationship set; detecting whether each target object relationship corresponding to the group subgraph is in the available relationship set; marking the edges corresponding to the target object relationships not in the available relationship set as edges with abnormal analysis, and marking the group subgraph as a group subgraph with abnormal analysis.

[0105] For example, predefine the association relationships that each target group type can contain, and use them as the available association relationships corresponding to the target group type.

[0106] Detect whether each target object relationship corresponding to the group subgraph is in the available relationship set. If there is a target object relationship in the relationship mining result that is not in the available relationship set, it indicates that there is an abnormality in the target object relationship or the target group type in the relationship mining result. Mark the edge corresponding to the target object relationship as an edge with abnormal analysis, and mark the group subgraph as a group subgraph with abnormal analysis.

[0107] Illustratively, if the relationship mining model identifies that the target group type of the group subgraph is group A, and the available association relationships corresponding to group A are queried to include relationship a and relationship b. If the relationship mining model identifies that the association relationships between the nodes in the group subgraph include relationship a, relationship b, and relationship c, where relationship c does not belong to the available association relationships corresponding to group A, it indicates that relationship c is abnormally identified or the target group type of the group subgraph is abnormally identified.

[0108] In addition to the methods for detecting and identifying abnormal edges and graphs in the above embodiments, the following can also be done: The group relationship mining also obtains the group type corresponding to the group subgraph, uses the group type obtained by the group relationship mining as the initial group type, and uses the association relationships between the objects to be analyzed represented by the original edges in the group subgraph as the initial object relationships. Detect the matching degree between the target group type and the initial group type of the group subgraph, and / or detect the matching degree between the target object relationships between the objects to be analyzed in the group subgraph and the initial object relationships; Mark the group subgraph with a matching degree lower than the threshold between the target group type and the initial group type as an abnormally analyzed group subgraph, and / or mark the edges with a matching degree lower than the threshold between the target object relationships and the initial object relationships as abnormally analyzed edges.

[0109] For example, the initial group type corresponding to the group subgraph is the A1 type of group. If the target group type obtained by relationship mining based on the description text is the A2 type of group, and the matching degree between the A1 type of group and the A2 type of group is greater than the threshold, it indicates that the target group type is credible; If the target group type obtained by relationship mining based on the description text is the B1 type of group, and the matching degree between the A1 type of group and the B1 type of group is less than the threshold, it indicates that the target group type is not credible, and mark this group subgraph as an abnormally analyzed group subgraph.

[0110] Or, from the object relationship graph constructed based on the perception data, the association relationship between node 1 and node 2 in the group subgraph is the a1 type of relationship. If the target object relationship obtained by relationship mining based on the description text is the a2 type of relationship, and the matching degree between the a1 type of relationship and the a2 type of relationship is greater than the threshold, it indicates that the target object relationship is credible; If the target object relationship obtained by relationship mining based on the description text is the b1 type of group, and the matching degree between the a1 type of relationship and the b1 type of group is less than the threshold, it indicates that the target object relationship is not credible, and mark the edge between node 1 and node 2 as an abnormally analyzed edge.

[0111] Of course, the above two embodiments for detecting and identifying abnormal edges and graphs can also be combined to comprehensively detect and identify abnormal edges and graphs, that is, detect whether each target object relationship corresponding to the group subgraph is in the set of available relationships, and detect whether the matching degree between the target object relationship and the initial object relationship is greater than the threshold and / or detect whether the matching degree between the target object relationship and the initial object relationship is greater than the threshold, and screen out the abnormal edges and graphs.

[0112] By judging the output of the relationship mining model through the above embodiments, it is possible to effectively and accurately screen out the edges or graphs that may be abnormal, and improve the accuracy of relationship mining.

[0113] Mark the abnormal edges and graphs.

[0114] In some embodiments, the group subgraph marked with abnormal recognition or the description text of the edge with abnormal recognition can be re - input into the relationship mining model for reasoning. At this time, it can be set that in addition to outputting the object relationship mining result, the relationship mining model also outputs the reasoning process, and the reasoning process is described in natural language (such as setting the reasoning process to be described with statements similar to first, then, and finally) for facilitating abnormal analysis.

[0115] If the new object relationship mining result is consistent with the original object relationship mining result, it is determined that the corresponding edge or group subgraph is abnormal.

[0116] In some embodiments, the method further includes: regarding the group subgraph marked as abnormal analysis and the edge marked as abnormal analysis as abnormal graph information; displaying the abnormal graph information on the analysis result display page; receiving the judgment result input for the abnormal graph information on the analysis result display page, where the judgment result is used to represent whether the abnormal graph information is abnormal; modifying the mark corresponding to the abnormal graph information based on the judgment result.

[0117] Exemplarily, please refer to Figure 4 , Figure 4 FIG. is a schematic diagram of the analysis result display page shown in an exemplary embodiment of the present application. The analysis result display page is used to display the object relationship mining results judged as normal and the object relationship mining results judged as abnormal, that is, to display the group subgraph marked as abnormal analysis and the edge marked as abnormal analysis, and highlight the edges or group subgraphs with abnormalities. For example, the abnormal edges are highlighted in red. Users can click on the abnormal edges to view the relevant information and reasoning process of the edges, and can also receive the judgment result input by the user for the abnormal graph information on the analysis result display page. The judgment result is used to represent whether the abnormal graph information is abnormal; modifying the mark corresponding to the abnormal graph information based on the judgment result to obtain a more accurate object relationship mining result.

[0118] The object relationship mining method provided by the present application obtains the perception data of each object to be analyzed by acquiring the attribute data and / or trajectory data corresponding to each object to be analyzed; constructs an object relationship graph based on the perception data of each object to be analyzed. The nodes in the object relationship graph represent the objects to be analyzed, and the edges in the object relationship graph represent the association relationships between the objects to be analyzed; converts the information corresponding to the nodes and / or edges in the object relationship graph into natural language to obtain the description text corresponding to the object relationship graph; inputs the description text corresponding to the object relationship graph into the trained relationship mining model to obtain the object relationship mining result output by the relationship mining model, so as to convert the structured information contained in the object relationship graph into natural language, fully mine the deep semantic information of the perception data, and combine the structured information of the object relationship graph and the natural language of the deep - level semantic information to improve the accuracy of the finally mined object relationship mining result.

[0119] Figure 5 is a block diagram of an object relationship mining device shown in an exemplary embodiment of the present application. As Figure 5 shown, the exemplary object relationship mining device 500 includes:

[0120] A data acquisition module 510, configured to acquire attribute data and / or trajectory data corresponding to each object to be analyzed, and obtain the perception data of each object to be analyzed;

[0121] A graph construction module 520, configured to construct an object relationship graph based on the perception data of each object to be analyzed. The nodes in the object relationship graph represent the objects to be analyzed, and the edges in the object relationship graph represent the association relationships between the objects to be analyzed;

[0122] A language conversion module 530, configured to convert the information corresponding to the nodes and / or edges in the object relationship graph into natural language, and obtain a description text corresponding to the object relationship graph;

[0123] A relationship mining module 540, configured to input the description text corresponding to the object relationship graph into a trained relationship mining model, and obtain an object relationship mining result output by the relationship mining model.

[0124] It should be noted that the object relationship mining device provided in the above embodiment and the object relationship mining method provided in the above embodiment belong to the same concept. The specific manners in which each module and unit perform operations have been described in detail in the method embodiment, and will not be repeated here. In practical applications, the object relationship mining device provided in the above embodiment can, according to needs, allocate the above functions to different functional modules, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above. This is not limited herein.

[0125] Please refer to Figure 6 , Figure 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application. The electronic device 600 includes a memory 6 and a processor 602. The processor 602 is configured to execute program instructions stored in the memory 601 to implement the steps in any of the above object relationship mining method embodiments. In a specific implementation scenario, the electronic device 600 may include, but is not limited to: a microcomputer, a server. In addition, the electronic device 600 may also include mobile devices such as a laptop computer, a tablet computer, etc., which are not limited herein.

[0126] Specifically, the processor 602 is used to control itself and the memory 601 to implement the steps in any of the above-described method embodiments for object relationship mining. The processor 602 may also be referred to as a Central Processing Unit (CPU). The processor 602 may be an integrated circuit chip with the ability to process signals. The processor 602 may also be a general-purpose processor, a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. Additionally, the processor 602 may be implemented jointly by integrated circuit chips.

[0127] Please refer to Figure 7 , Figure 7 FIG. is a schematic structural diagram of an embodiment of the computer-readable storage medium of the present application. The computer-readable storage medium 700 stores program instructions 710 that can be run by a processor, and the program instructions 710 are used to implement the steps in any of the above-described method embodiments for object relationship mining.

[0128] In some embodiments, the functions or modules included in the device provided by the present disclosure embodiment can be used to execute the methods described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0129] The above descriptions of each embodiment tend to emphasize the differences between the embodiments. Their similarities or similarities can be referred to each other. For the sake of brevity, they will not be repeated in this article.

[0130] In several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in electrical, mechanical or other forms.

[0131] In addition, each functional unit in various embodiments of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, may be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods in various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs that can store program codes.

Claims

1. An object relationship mining method, characterized in that: The method comprises: Acquire attribute data and / or trajectory data corresponding to each object to be analyzed, and obtain perception data of each object to be analyzed; Building an object relationship graph based on the perception data of each object to be analyzed, wherein the nodes in the object relationship graph represent the objects to be analyzed, and the edges in the object relationship graph represent the association relationships between the objects to be analyzed; Converting information corresponding to nodes and / or edges in the object relationship graph into natural language to obtain a description text corresponding to the object relationship graph; The description text corresponding to the object relationship diagram is input into the trained relationship mining model to obtain the object relationship mining result output by the relationship mining model.

2. The method according to claim 1, characterized in that The converting information corresponding to nodes and / or edges in the object relationship graph into natural language to obtain description text corresponding to the object relationship graph includes: Performing group relationship mining on the object relationship graph to divide the object relationship graph into a plurality of group subgraphs; The information corresponding to the nodes and / or edges in each group subgraph is converted into natural language to obtain description text corresponding to each group subgraph.

3. The method according to claim 2, characterized in that The converting the information corresponding to the nodes and / or edges in each group subgraph into natural language to obtain the description text corresponding to each group subgraph includes: Converting attribute information corresponding to each node and / or each edge in the group subgraph into a natural language statement; Splicing the natural language sentences corresponding to the attribute information to obtain node description texts corresponding to the nodes and / or edge description texts corresponding to the edges; The node description text corresponding to each node and / or the edge description text corresponding to each edge are summarized and analyzed to obtain a summary description text corresponding to the group subgraph.

4. The method according to claim 2, characterized in that The step of inputting the description text corresponding to the object relationship diagram into the trained relationship mining model to obtain the object relationship mining result output by the relationship mining model includes: The descriptive text corresponding to the group subgraph is input into the trained relationship mining model, the group type corresponding to the group subgraph output by the relationship mining model is used as the target group type, and the association relationship between each object to be analyzed in the group subgraph output by the relationship mining model is used as the target object relationship.

5. The method according to claim 4, characterized in that The method further comprises: Obtaining preset usable association relationships corresponding to the target group type to obtain a usable relationship set; Detecting whether all target object relationships corresponding to the group subgraph are in the usable relationship set; The edge corresponding to the target object relationship that is not in the usable relationship set is marked as an analysis abnormality edge, and the group subgraph is marked as an analysis abnormality group subgraph.

6. The method according to claim 4, characterized in that The group relationship mining further obtains the group type corresponding to the group subgraph, uses the group type obtained by the group relationship mining as the initial group type, and uses the association relationship between each object to be analyzed represented by the original edge in the group subgraph as the initial object relationship; the method further includes: Detecting a degree of match between a target group type and an initial group type in the group subgraph, and / or detecting a degree of match between a target object relationship and an initial object relationship between each object to be analyzed in the group subgraph; A group subgraph whose matching degree between the target group type and the initial group type is lower than a threshold is marked as an analysis abnormal group subgraph, and / or an edge whose matching degree between the target object relationship and the initial object relationship is lower than a threshold is marked as an analysis abnormal edge.

7. The method according to any one of claims 5 or 6, characterized in that The method further comprises: The group subgraph marked as analysis anomaly and the edges of analysis anomaly are used as anomaly graph information; Display the abnormal graph information on the analysis result display page; receiving a judgment result input on the abnormal graph information in the analysis result display page, wherein the judgment result is used to indicate whether the abnormal graph information has an abnormality; The mark corresponding to the abnormal graph information is modified based on the judgment result.

8. The method according to claim 4, characterized in that The relationship mining model stores natural language definitions corresponding to a plurality of candidate group types and a plurality of candidate object relationships; the description text corresponding to the group subgraph is input into the trained relationship mining model, the relationship mining model outputs the group type corresponding to the group subgraph to obtain the target group type, and the relationship mining model outputs the association relationship between each object to be analyzed in the group subgraph to obtain the target object relationship, including: Calculating the matching degree between the description text corresponding to the object relationship graph and the natural language definition corresponding to each candidate group type using the relationship mining model, and selecting the candidate group type whose matching degree meets the preset conditions as the target group type; and The relationship mining model is used to calculate the matching degree between the description text corresponding to the object relationship graph and the natural language definition corresponding to each candidate object relationship, and the candidate object relationship whose matching degree meets the preset conditions is selected as the target object relationship.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, and the processor is used to execute program instructions stored in the memory to implement the steps in the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores program instructions, and the program instructions can be executed by a processor to implement the steps in the method according to any one of claims 1 to 8.

Images