Mobile terminal intelligent map construction and self-adaptive visualization method and system

Abstract

The invention provides a mobile terminal intelligent map construction and self-adaptive visualization method and system, and the method comprises the steps: collecting various source data, such as texts, images, audios and videos, through technologies such as crawlers; processing and integrating the obtained different types of heterogeneous data to form structured data and semi-structured data; training a knowledge extraction model, and reading structured and semi-structured data to perform entity recognition and relationship extraction; writing an automatic atlas construction program, and storing a construction result in a database; and acquiring related parameters of the equipment to write a self-adaptive visual program, and displaying associated information of the entity. According to the method, efficient atlas construction and self-adaptive visual display at the mobile terminal are realized through integrated method design and intelligent technical means. The method not only improves the construction efficiency and quality of the knowledge graph, but also provides a more convenient and intuitive information acquisition mode for the user, and improves the understanding and analysis efficiency of the user on complex information.

Description

Technical Field

[0001] The present invention relates to the field of knowledge graphs, and in particular to a mobile intelligent graph construction and adaptive visualization method and system. Background Art

[0002] With the popularity of mobile smart devices and the continuous improvement of their performance, people are increasingly querying and analyzing various information on mobile devices. As an effective means of structured representation of complex relational data, knowledge graphs have been widely used in many fields. However, most traditional graph construction methods rely on complex computing environments and a large amount of manual intervention, making it difficult to achieve efficient construction and display on mobile devices. Existing mobile information display methods are often relatively simple and difficult to meet users' needs for understanding and analyzing complex information. Especially when dealing with a large amount of related information, traditional display methods can easily lead to information confusion, difficulty in reading and understanding.

[0003] At present, the knowledge graph construction process is relatively mature, and the methods used are also diverse. The main construction process includes data collection, knowledge extraction, and knowledge graph construction. The most core methods involved are knowledge extraction methods and knowledge graph construction methods. Knowledge extraction includes entity recognition and relationship extraction. The mainstream methods include matching predefined rules to identify entities and extract relationships from text, using NLP (natural language processing) technology to analyze text semantics to identify entities and extract relationships, and using machine learning algorithms to train models to identify entities and extract relationships in text; the mainstream methods for knowledge graph construction include manual construction, automatic construction, semi-automatic construction, etc.

[0004] On the other hand, with the booming development of algorithms such as machine learning and the improvement of hardware device performance, the proposed mobile intelligent graph construction and adaptive visualization method and system are more feasible. The characteristics of mobile knowledge graph construction and visualization are short cycle and high timeliness, which requires reducing the reliance on manual intervention. Therefore, an automated and intelligent construction method is needed to meet the needs of mobile visualization. Summary of the invention

[0005] The purpose of the present invention is to propose a mobile terminal intelligent graph construction and adaptive visualization method and system to solve the shortcomings of existing mobile terminal information display methods and improve users' understanding and analysis efficiency of complex information.

[0006] The technical solution to achieve the purpose of the present invention is: a method for constructing and adaptively visualizing a mobile intelligent graph, comprising the following steps:

[0007] Step 1: Collect data from multiple sources, including open source and closed source channels, in the form of text, images, audio, and video, and integrate them into structured data and semi-structured data;

[0008] Step 2: Train the knowledge extraction model to perform entity recognition and relationship extraction on structured and semi-structured data to form a triple structure. Use entities as nodes and relationships as edges to build a mobile intelligent graph. Entities include people, institutions, parties, places, positions, schools, majors, events, honors, and achievements. Relationships include appointment, participation, political party, graduation from, education, honor, family, social interaction, influence, birthplace, residence, and others.

[0009] Step 3, obtaining relevant parameters of the mobile device, including screen width and height, screen brightness, and screen refresh frame rate; establishing a color template, predefining color RGB values ​​for entities and relationships, and forming a visualization color correspondence table of entities and relationships; combining the relevant parameters of the mobile device with the visualization color correspondence table of entities and relationships to obtain visualization template parameters;

[0010] Step 4: Call the graph retrieval interface to obtain the number of relationship categories and the number of entity nodes under each relationship, dynamically calculate the layout according to the visualization template parameters, and use classification aggregation and layered rendering to draw the mobile smart graph.

[0011] Furthermore, in step 1, data from multiple sources, including open source channels and closed source channels, in the form of text, images, audio, and video, are collected and integrated into structured data and semi-structured data. The specific method is as follows:

[0012] (1) Open source channels

[0013] Collect open source data from the Internet through crawler technology, perform data preprocessing, clean text data, remove special characters and special codes, special characters include "%", ":", "]", "[", "\", special codes include "ASCII"; for image data, regularize the naming method and unify the image file format, regularize non-"*.jpg, *.jpeg, *.png" formats into "*.jpg, *.jpeg, *.png", and save them in the naming method of "name + date + number"; for audio data and video data, regularize the file naming method and save them in the method of "name + date + number";

[0014] (2) Closed-source channels

[0015] Closed-source data is classified into paper data and electronic data. For paper data, OCR recognition is performed to form text and then processed into paragraphs. For electronic data, word documents are formed into semi-structured paragraph texts, and excel tables are structured according to fields.

[0016] (3) Integration

[0017] Integrate various types of data from the above open source channels and closed source channels, classify them according to file format types, and write scripts to parse the collected web page data in Html format files to generate structured json files; text data is divided into paragraphs and sentences and stored to generate semi-structured text txt files; image files, audio and video files and video files are stored according to file names and address URLs to generate structured json files.

[0018] Furthermore, in step 2, a knowledge extraction model is trained to perform entity recognition and relationship extraction on structured data and semi-structured data to form a triple structure, and a mobile intelligent graph is established with entities as nodes and relationships as edges. The specific method is as follows:

[0019] The knowledge extraction model uses the Qwen2-0.5b-Instruct pre-trained model to perform entity recognition and relationship extraction on structured and semi-structured data. The input is structured and semi-structured text data, and the output is entity recognition results and relationship extraction results.

[0020] The Qwen2-0.5b-Instruct pre-trained model includes 4 dense models and 1 expert mixture model. The dense model parameter range includes 0.5B, 1.5B, 7B, and 72B, and the expert mixture model parameter range includes 57B-A14B;

[0021] (2) Store the extracted entities in the database in the form of nodes to form an entity set: V s = {V1, V2, ... V n}, each entity V n It also contains a collection of attribute information:

[0022] Organize the extracted relations in the form of edges, build the relations between the head entity and the tail entity, and form a relation set: E s ={E1, E2, ... E n}, each relation E n It also contains a collection of attribute information:

[0023] For the constructed entity set V s and the relation set E s , create the entity table "vertex" and the relationship table "edge" in the mongodb database, and build the entity index and relational indexes , encapsulates the retrieval interface API and provides support for the visualization of intelligent graphs on mobile terminals.

[0024] Furthermore, the entity set: V s= {V1, V2, ... V n}, each entity V n It also contains a collection of attribute information: in:

[0025] The entity "person" contains the attribute information "Chinese name", "English name", "alias", "nationality", "gender", "ethnicity", and "date of birth";

[0026] The entity "Organization" contains the attribute information "Chinese name", "English name", and "Introduction";

[0027] The entity "Party" contains the attribute information "Chinese name", "English name", and "Introduction";

[0028] The entity "place" contains the attribute information "Chinese name", "English name", "introduction", and "geographic coordinates";

[0029] The entity "Position" contains the attribute information "Chinese name", "English name", and "Job content";

[0030] The entity "school" contains attribute information "Chinese name", "English name", and "introduction";

[0031] The entity "major" contains the attribute information "Chinese name", "English name", and "major course";

[0032] The entity "event" contains the attribute information "event location", "event content", "occurrence time", and "end time";

[0033] The entity "honor" contains attribute information "honor type" and "acquisition time";

[0034] The entity "achievement" includes attribute information "achievement content" and "acquisition time".

[0035] Furthermore, the relation set: E s ={E1, E2, ... E n}, each relation E n It also contains a collection of attribute information: in:

[0036] The relationship "Job Position" includes the head entity "Person" and the tail entity "Organization" and the attribute information "Position Held", "Job Start Time", and "Job End Time";

[0037] The relationship "participation" includes the head entity "person" and the tail entity "event" and the attribute information "participation time", "participation location", and "participation content";

[0038] The relationship "Political Party" includes the head entity "Person" and the tail entity "Party" and the attribute information "Participation Time" and "Participation Location";

[0039] The relationship "Graduated from" includes the head entity "Person" and the tail entity "Major" and the attribute information "Start Time", "Learning Place", "End Time", and "Degree Obtained";

[0040] The relationship "education" includes the head entity "person" and the tail entity "school" and the attribute information "enrollment time", "education location", and "school leaving time";

[0041] The relationship "honor" includes the head entity "person" and the tail entity "honor" and the attribute information "honor type", "acquisition location", and "acquisition time";

[0042] The relationship "family affection" includes the head entity "person" and the tail entity "person" and the attribute information "husband and wife", "father and son", and "father and daughter";

[0043] The relationship "social" includes the head entity "person" and the tail entity "person" and the attribute information "friends" and "colleagues";

[0044] The relationship “influence” includes the head entity “person” and the tail entity “achievement” and the attribute information “acquisition time” and “influence content”;

[0045] The relationship "birthplace" includes the head entity "person" and the tail entity "place" and the attribute information "birthtime";

[0046] The relationship "residence" includes the head entity "person" and the tail entity "place" and the attribute information "residence time";

[0047] The relationship "other" includes the head entity "person" and the tail entity "event" and the attribute information "start time" and "other content".

[0048] Further, in step 3, the relevant parameters of the mobile device are obtained, including screen width and height, screen brightness, and screen refresh frame rate. The specific method is as follows:

[0049] (1) Screen width and height:

[0050] Android devices use the DisplayMetrics class to obtain the width and height of the screen, initialize the class object DisplayMetrics displayMetrics = new DisplayMetrics(); then get the value of the object according to the getMetrics function getWindowManager().getDefaultDisplay().getMetrics(displayMetrics); finally get the width and height parameters int width = displayMetrics.widthPixels; int height = displayMetrics.heightPixels;

[0051] IOS devices use the UIScreen class to get the width and height of the current screen, let screenWidth = UIScreen.main.bounds.size.width; let screenHeight = UIScreen.main.bounds.size.height;

[0052] (2) Screen brightness:

[0053] Android devices use the WindowManager class to obtain the current screen brightness, initialize the class object WindowManager wm = (WindowManager) context.getSystemService (Context.WI NDOW_SERVICE); then use the getBrightness() function to obtain the screen brightness int brightness = wm.getDefaultDisplay().getBrightness();

[0054] IOS devices use the UIScreen class to get the current screen brightness, let screenBrightness = UIScreen.mainScreen().brightness;

[0055] (3) Screen refresh frame rate refreshrate:

[0056] Android devices use the DisplayMetrics class to obtain the refresh frame rate of the screen, initialize the class object DisplayMetrics displayMetrics = new DisplayMetrics(); then obtain the refresh frame rate through the getRefreshRate() function, getWindowManager().getDefaultDisplay().getRefreshRate();

[0057] The iOS device uses the UIScreen class to obtain the refresh frame rate of the current screen, then initializes the class object and obtains the refresh frame rate, let screen = UIScreen.mainScreen(); let refreshRate:Double = screen.nativeOrientation == .portrait? 60:120.

[0058] Further, in step 3, relevant parameters of the mobile device are obtained, including screen width and height, screen brightness, and screen refresh frame rate; a color template is established to predefine color RGB values ​​for entities and relationships, respectively, to form a visual color correspondence table of entities and relationships. The specific method is as follows:

[0059] (1) The entities include the following:

[0060] People, organizations, parties, places, positions, schools, majors, events, honors, and achievements; the color RGB value correspondence table is as follows:

[0061] The character's color RGB value is set to (215, 135, 105);

[0062] The color RGB value of the organization is set to (150, 125, 60);

[0063] The RGB value of the party color is set to (105, 150, 60);

[0064] The RGB value of the location color is set to (75, 150, 60);

[0065] The color RGB value of the position is set to (35, 125, 95);

[0066] The RGB value of the school color is set to (85, 75, 125);

[0067] The professional color RGB value is set to (115, 75, 165);

[0068] The event color RGB value is set to (125, 85, 95);

[0069] The RGB value of the honor color is set to (215, 55, 55);

[0070] The RGB value of the achievement color is set to (65, 25, 190);

[0071] (2) The relationships are as follows:

[0072] Position, Participation, Political Party, Graduated from, Education, Honor, Family, Social, Influence, Birthplace, Residence, Others; the color RGB value correspondence table is as follows:

[0073] The RGB value of the color in use is set to (255, 15, 15);

[0074] The RGB values ​​of the participating colors are set to (15, 255, 15);

[0075] The RGB value of the political party color is set to (105, 255, 105);

[0076] The RGB value of the graduated color is set to (255, 255, 0);

[0077] The color RGB value of education is set to (185, 175, 125);

[0078] The RGB value of the honor color is set to (55, 25, 85);

[0079] The RGB value of the color of affection is set to (205, 25, 45);

[0080] The social color RGB value is set to (205, 115, 85);

[0081] The affected color RGB value is set to (85, 65, 45);

[0082] The RGB value of the birthplace color is set to (205, 95, 95);

[0083] The RGB value of the color of the residence is set to (90, 30, 30);

[0084] The RGB values ​​of other colors are set to (120, 150, 160);

[0085] Furthermore, in step 4, the graph retrieval interface is called to obtain the number of relationship categories and the number of entity nodes under each relationship, and the layout is dynamically calculated according to the visualization template parameters. The mobile terminal intelligent graph is drawn by classification aggregation and layered rendering, where:

[0086] (1) The classification aggregation method is as follows:

[0087] When the total number of knowledge graph data nodes is large, the relationship is used as the classification target, the entities of the same relationship are aggregated and rendered in the form of nodes. Due to the limited visualization space, the entities of the same relationship category are set with specified threshold parameters. If the number of Node count >3, it is displayed in an aggregated form; if the number of Node count If the value is less than 3, it will be directly rendered and displayed; the entity node name display and attribute field information display are expressed in an explicit and implicit way, that is, the name is the only visual field, and the attribute field information is displayed as the field displayed after interaction;

[0088] (2) The layered rendering method is as follows:

[0089] In order to solve the problem of rendering a large number of nodes at the same time in the visual interface, the node rendering method is adopted to display the total number of nodes that need to be rendered under the current relationship category. count , calculate the total number of layers that need to be rendered according to the classified aggregated nodes count , and then calculate the number of rendering nodes of the current layer according to the number of layers When the total number of nodes under each type of relationship count >57 and the total number of layers count >2, the total number of layers rendered is calculated using the formula:

[0090] 40+5*(3+Layer count )*(Layer count -2)=Node count

[0091] That is:

[0092] First layer count＝1 hour,

[0093] Second Layer count＝2 hour,

[0094] The third layer count＝3 hour,

[0095] The fourth layer count＝4 hour, ...

[0097] Layer n count＝n hour,

[0098] Furthermore, it also includes:

[0099] Step 5: Use Cytoscape.js graph rendering and interactive visualization tools for display, and support interactive actions including: touch, slide, zoom, classification aggregation, topological space extension, parameter feedback, among which:

[0100] (1) The topological space is extended as follows:

[0101] Topological space extension includes: dynamically allocating space according to the display results of different entity nodes; extending the display range space to the corresponding direction according to the relationship connection between nodes; and realizing the circular extension of nodes;

[0102] (2) Parameter feedback is as follows:

[0103] Parameter feedback includes the width and height of the mobile device screen, screen brightness, screen refresh rate, and the custom RGB value of the color template.

[0104] A mobile terminal intelligent graph construction and adaptive visualization system implements the mobile terminal intelligent graph construction and adaptive visualization method to achieve mobile terminal intelligent graph construction and adaptive visualization.

[0105] Compared with the prior art, the present invention has the following significant advantages: 1) It realizes the integration of multi-source data and improves the breadth and depth of the knowledge graph; 2) Through intelligent graph construction and adaptive visualization of mobile devices, complex information is presented to users in an intuitive and clear manner, helping users to better understand and analyze information; 3) It enhances the user experience, provides a variety of interaction methods and dynamic adjustment functions, and enables users to browse and operate graphs more conveniently, thereby enhancing the user experience; 4) It adapts to different devices and can automatically adapt to different mobile devices, providing good display effects for both small-screen mobile phones and large-screen tablets. BRIEF DESCRIPTION OF THE DRAWINGS

[0106] Figure 1 It is a flow chart of the method for constructing and adaptively visualizing a mobile terminal intelligent graph according to the present invention;

[0107] Figure 2 It is a flowchart of multi-source data integration;

[0108] Figure 3 It is a schematic diagram of the principle of knowledge extraction;

[0109] Figure 4 It is a schematic diagram of the construction and storage of knowledge graphs;

[0110] Figure 5 It is a schematic diagram of adaptive visualization for mobile devices. DETAILED DESCRIPTION

[0111] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0112] The present invention provides a mobile terminal intelligent graph construction and adaptive visualization method and system thereof, which aims to construct a high-quality knowledge graph in an automated and intelligent manner, and obtain specific parameters of mobile devices to achieve adaptive visualization of mobile device graphs. The present invention uses the Qwen2-0.5B open source pre-trained large model as the knowledge extraction model, stores the extracted triple structured information in a NoSQL type MongoDB database, and uses Cytoscape.js as a graph visualization tool for rendering and interaction with mobile devices, and introduces in detail the process of the mobile terminal intelligent graph construction and adaptive visualization method of the present invention. Figure 1 Shown is the overall flow chart of the method of the present invention, Figure 2 The figure shows the flow chart of multi-source data integration. Figure 3 The figure shows the principle diagram of knowledge extraction. Figure 4 The following is a schematic diagram of the knowledge graph construction and storage. Figure 5 Shown is the schematic diagram of adaptive visualization for mobile devices.

[0113] A mobile terminal intelligent graph construction and adaptive visualization method, the main execution steps are as follows:

[0114] Step 1: Use crawlers and other technologies to collect data from multiple sources, including open-source and closed-source channels. The data forms include text, images, audio, video, etc. It should be noted that open-source channels are Internet data, and closed-source channels are internal data. The different data obtained are processed and integrated to form structured data and semi-structured data.

[0115] A. Open source channels

[0116] Open source data is collected through crawler technology, and text data is cleaned and preprocessed to remove noise and invalid information such as special characters and special codes, such as "%", ":", "]", "[", "\", and special codes such as "ASCII"; for image data, the naming method is regularized and the image file format is unified. For example, *.webp, *.gif, *.tiff and other non-"*.jpg, *.jpeg, *.png" formats are regularized into "*.jpg, *.jpeg, *.png", and the naming method is saved in the "name + date + number" method; for audio data and video data, the naming method of the regularized files is saved in the "name + date + number" method.

[0117] B. Closed-source channels

[0118] Closed-source data is classified into paper data and electronic data. Paper data is subjected to OCR recognition to form text and then processed into paragraphs. Electronic data is processed according to word and excel types, with word documents forming semi-structured paragraph text and excel tables being structured according to fields.

[0119] Integrate various types of data from the above open source and closed source channels, and manage and store them according to file format types.

[0120] Step 2: First, analyze the data set formed in step 1 to extract data categories, such as people, institutions, parties, places, positions, schools, majors, events, honors, and achievements. Then, based on the Qwen2-0.5B open source pre-trained large model, the 0.5B parameter open source pre-trained large model in the Qwen2 series is designed to provide researchers and developers with a powerful language model foundation. Perform the first entity recognition and relationship extraction. Entity recognition includes people, institutions, parties, places, positions, schools, majors, events, honors, and achievements; relationship extraction includes appointment, participation, political party, graduation, education, honor, family, social, influence, birthplace, residence, and others. Verify the recognition and extraction results, and use the verified results as training data sets and verification data sets. Next, fine-tune the large model based on the training data set, and then perform entity recognition and relationship extraction again and supplement the verified results as training data sets and verification data sets. Repeat the above steps until the accuracy is not significantly improved. Finally, the iterated knowledge extraction model is used to perform entity recognition and relationship extraction, and the entities and relationships are output in the form of triples.

[0121] Step 3: The triple structure data output in step 2 is divided into nodes (entities) and edges (relationships) for storage. Nodes represent entities, and edges represent relationships between entities. Each node and edge is given corresponding attributes, such as name, description, weight, etc. New tables "vertex" and "edge" are created in the MongoDB database to store nodes and edges respectively. The corresponding entity attributes and relationship attributes are stored as a set of subsidiary fields in the corresponding tables.

[0122] Forming entity collection: V s = {V1, V2, ... V n}, each entity V n It also contains a collection of attribute information: For example, the entity "person" contains the attribute information "Chinese name", "English name", "alias", "nationality", "gender", "ethnicity", and "date of birth". The entity "institution" contains the attribute information "Chinese name", "English name", and "introduction"; the entity "party" contains the attribute information "Chinese name", "English name", and "introduction"; the entity "place" contains the attribute information "Chinese name", "English name", "introduction", and "geographic coordinates"; the entity "position" contains the attribute information "Chinese name", "English name", and "work content"; the entity "school" contains the attribute information "Chinese name", "English name", and "introduction"; the entity "major" contains the attribute information "Chinese name", "English name", and "major course"; the entity "event" contains the attribute information "event location", "event content", "occurrence time", and "end time"; the entity "honor" contains the attribute information "honor type" and "obtaining time"; the entity "achievement" contains the attribute information "achievement content" and "obtaining time";

[0123] Relationship set: E s ={E1, E2, ... E n}, each relation E n It also contains a collection of attribute information: For example, the relationship "holding a post" contains the head entity "person" and the tail entity "institution" and the attribute information "position held", "start time of serving", "end time of serving"; the relationship "participation" contains the head entity "person" and the tail entity "event" and the attribute information "participation time", "participation location", "participation content"; the relationship "political party" contains the head entity "person" and the tail entity "party" and the attribute information "participation time", "participation location"; the relationship "graduated from" contains the head entity "person" and the tail entity "major" and the attribute information "start time", "study location", "end time", "degree obtained"; the relationship "education" contains the head entity "person" and the tail entity "school" and the attribute information "enrollment time", "education location", "leaving time"; the relationship "honor" contains the head entity "person" and the tail entity " The entity "Honor" and the attribute information "Honor type", "Obtaining place", "Obtaining time"; the relationship "Family", including the head entity "Person" and the tail entity "Person" and the attribute information "Husband and wife", "Father and son", "Father and daughter"; the relationship "Social", including the head entity "Person" and the tail entity "Person" and the attribute information "Friends", "Colleagues"; the relationship "Influence", including the head entity "Person" and the tail entity "Achievement" and the attribute information "Obtaining time", "Influence content"; the relationship "Birthplace", including the head entity "Person" and the tail entity "Place" and the attribute information "Birth time"; the relationship "Residence", including the head entity "Person" and the tail entity "Place" and the attribute information "Residence time"; the relationship "Other", including the head entity "Person" and the tail entity "Event" and the attribute information "Start time", "Other content".

[0124] Finally, according to the constructed entity set Vs and relationship set Es, the entity table "vertex" and relationship table "edge" are established in the mongodb database and the entity table index is constructed. and relational table indexes Then the retrieval interface API is encapsulated to provide support for adaptive graph data visualization.

[0125] Step 4: Get the relevant parameters of the mobile device. The parameters are as follows:

[0126] Screen width and height (width, height):

[0127] Android devices use the DisplayMetrics class to obtain the width and height of the screen, initialize the class object DisplayMetrics displayMetrics = new DisplayMetrics(); then get the value of the object according to the getMetrics function getWindowManager().getDefaultDisplay().getMetrics(displayMetrics); finally get the width and height parameters int width = displayMetrics.widthPixels; int height = displayMetrics.heightPixels;

[0128] IOS devices use the UIScreen class to get the width and height of the current screen, let screenWidth = UIScreen.main.bounds.size.width; let screenHeight = UIScreen.main.bounds.size.height;

[0129] Screen brightness:

[0130] Android devices use the WindowManager class to obtain the current screen brightness, initialize the class object WindowManager wm = (WindowManager) context.getSystemService (Context.WI NDOW_SERVICE); then use the getBrightness() function to obtain the screen brightness int brightness = wm.getDefaultDisplay().getBrightness();

[0131] IOS devices use the UIScreen class to get the current screen brightness, let screenBrightness = UIScreen.mainScreen().brightness;

[0132] Screen refresh frame rate (refreshrate):

[0133] Android devices use the DisplayMetrics class to obtain the refresh frame rate of the screen, initialize the class object DisplayMetrics displayMetrics = new DisplayMetrics(); then obtain the refresh frame rate through the getRefreshRate() function, getWindowManager().getDefaultDisplay().getRefreshRate();

[0134] IOS devices use the UIScreen class to obtain the refresh frame rate of the current screen, then initialize the class object and obtain the refresh frame rate, let screen = UIScreen.mainScreen(); let refreshRate:Double = screen.nativeOrientation == .portrait? 60:120;

[0135] Step 5: Create a color template. According to the entity category, predefine the color RGB value for each node, and the RGB value of each node is editable and customizable. This forms a corresponding table between entity nodes and their visual colors, and the same is true for relationship categories.

[0136] A. The entities include the following:

[0137] People, organizations, political parties, places, positions, schools, majors, events, honors, and achievements; the color RGB value correspondence table is as follows:

[0138] The character's color RGB value is set to (215, 135, 105);

[0139] The color RGB value of the organization is set to (150, 125, 60);

[0140] The RGB value of the political party color is set to (105, 150, 60);

[0141] The RGB value of the location color is set to (75, 150, 60);

[0142] The color RGB value of the position is set to (35, 125, 95);

[0143] The RGB value of the school color is set to (85, 75, 125);

[0144] The professional color RGB value is set to (115, 75, 165);

[0145] The event color RGB value is set to (125, 85, 95);

[0146] The RGB value of the honor color is set to (215, 55, 55);

[0147] The RGB value of the achievement color is set to (65, 25, 190);

[0148] B. The relationships are as follows:

[0149] Position, Participation, Party, Graduated from, Education, Honor, Family, Social, Influence, Birthplace, Residence, Others; the color RGB value corresponding table is as follows:

[0150] The RGB value of the color in use is set to (255, 15, 15);

[0151] The RGB values ​​of the participating colors are set to (15, 255, 15);

[0152] The RGB value of the party color is set to (105, 255, 105);

[0153] The RGB value of the graduated color is set to (255, 255, 0);

[0154] The color RGB value of education is set to (185, 175, 125);

[0155] The RGB value of the honor color is set to (55, 25, 85);

[0156] The RGB value of the color of affection is set to (205, 25, 45);

[0157] The RGB value of the social color is set to (205, 115, 85);

[0158] The affected color RGB value is set to (85, 65, 45);

[0159] The RGB value of the birthplace color is set to (205, 95, 95);

[0160] The RGB value of the color of the residence is set to (90, 30, 30);

[0161] The RGB values ​​of other colors are set to (120, 150, 160);

[0162] Step 6: Using the parameter values ​​obtained in steps 4 and 5, you can create a visualization template parameter library Tep s ={p width , p height , p brightness , p refreshrate , p color}, where p width is the screen width of the mobile device, p height is the height of the mobile device screen, pbrightness is the screen brightness of the mobile device, p redreshrate is the screen refresh rate of the mobile device, p color The table below is the color correspondence table for entities and relationships. The visualization template parameter library changes according to the changes of mobile devices, and can also change according to user-defined content.

[0163] Step 7: Get the number of relationship categories by calling the graph retrieval interface API. count And the number of entity nodes under each relationshipNode count , and according to the visualization template parameter Tep s Dynamically calculate the layout to achieve node and edge rendering using classification aggregation and layered rendering. The layout and core parameters involved include classification aggregation parameters and layered rendering parameters.

[0164] The classification aggregation parameters are set as follows:

[0165] When the total number of knowledge graph data nodes is large, the relationship is used as the classification target, and the entities of the same relationship are aggregated and rendered as nodes. Due to the limited visualization space, the entities of the same relationship category are set with a specified threshold parameter. If the number of Node count >3, it is displayed in an aggregated form; if the number of Node count If the value is less than 3, it will be directly rendered and displayed. The entity node name and attribute field information are displayed in an explicit and implicit way, that is, the name is the only visual field, and the attribute field information is displayed as the field displayed after interaction;

[0166] The layered rendering parameters are set as follows:

[0167] In order to solve the problem of rendering a large number of nodes at the same time in the visual interface, the node rendering method is adopted. Get the total number of nodes that need to be rendered under the current relationship categoryNode count , calculate the total number of layers that need to be rendered according to the classified aggregated nodes count , and then calculate the number of rendering nodes of the current layer according to the number of layers When the total number of nodes under each type of relationship count >57 and the total number of layers count >2, the total number of layers rendered is calculated using the formula:

[0168] 40+5*(3+Layer count )*(Layer count -2)=Node count

[0169] That is:

[0170] First layer count＝1 hour,

[0171] Second Layer count＝2 hour,

[0172] The third layer count＝3 hour,

[0173] The fourth layer count＝4 hour, ...

[0175] Layer n count＝n hour,

[0176] Step 8: After the relevant parameters are calculated in step 7, the Cytoscape.js graph rendering and interactive visualization tools are used to display and support interactive actions including: touch, slide, zoom, classification aggregation, topological space extension, and parameter feedback. Among them, the touch, slide, zoom, and classification aggregation interactive actions are not described in detail. Next, the topological space extension and parameter feedback are explained respectively.

[0177] The topological space extension interaction is as follows:

[0178] Due to the limited space on the mobile device interface, a topological space extension method is used when rendering and visualizing a large number of graph node associations. Specifically, it includes: dynamically allocating space according to the display results of different entity nodes; extending the display range space to the corresponding direction according to the relationship between nodes; and realizing the circular extension of nodes;

[0179] The parameter feedback is as follows:

[0180] The parameter feedback is mainly for the content in the calculation template in this method, the width and height of the mobile device screen, the screen brightness, the screen refresh rate, and the custom RGB value of the color template.

[0181] The present invention proposes a mobile intelligent graph construction and adaptive visualization system, which obtains knowledge sources through multi-source data integration, integrates knowledge extraction models and algorithms, realizes automatic and intelligent construction of knowledge graphs, and designs an adaptive visualization system to display it on the mobile device application interface.

[0182] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0183] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A mobile terminal intelligent graph construction and adaptive visualization method, characterized in that: The steps include: Step 1: Collect data from multiple sources, including open source and closed source channels, in the form of text, images, audio, and video, and integrate them into structured data and semi-structured data; Step 2: Train the knowledge extraction model to perform entity recognition and relationship extraction on structured and semi-structured data to form a triple structure. Use entities as nodes and relationships as edges to build a mobile intelligent graph. Entities include people, institutions, parties, places, positions, schools, majors, events, honors, and achievements. Relationships include appointment, participation, political party, graduation from, education, honor, family, social interaction, influence, birthplace, residence, and others. Step 3, obtaining relevant parameters of the mobile device, including screen width and height, screen brightness, and screen refresh frame rate; establishing a color template, predefining color RGB values ​​for entities and relationships, and forming a visualization color correspondence table of entities and relationships; combining the relevant parameters of the mobile device with the visualization color correspondence table of entities and relationships to obtain visualization template parameters; Step 4: Call the graph retrieval interface to obtain the number of relationship categories and the number of entity nodes under each relationship, dynamically calculate the layout according to the visualization template parameters, and use classification aggregation and layered rendering to draw the mobile smart graph.

2. The method for constructing and adaptively visualizing a mobile intelligent graph according to claim 1, characterized in that: Step 1: Collect data from multiple sources, including open source channels and closed source channels. The data forms include text, images, audio, and video, and integrate them into structured data and semi-structured data. The specific method is: (1) Open source channels Open source data is collected from the Internet through crawler technology, and data preprocessing is performed. Text data is cleaned to remove special characters and special codes, where special characters include "%", ":", "]", "[", "\", and special codes include "ASCII"; for image data, the naming method is standardized and the image file format is unified, and non-"*.jpg, *.jpeg, *.png" formats are standardized to "*.jpg, *.jpeg, *.png", and the naming method is saved in the "name + date + number" method; for audio data and video data, the naming method of the regularized file is saved in the "name + date + number" method; (2) Closed-source channels Closed-source data is classified into paper data and electronic data. For paper data, OCR recognition is performed to form text and then processed into paragraphs. For electronic data, word documents are formed into semi-structured paragraph texts, and excel tables are structured according to fields. (3) Integration Integrate various types of data from the above open source channels and closed source channels, classify them according to file format types, and write scripts to parse the collected web page data in Html format files to generate structured json files; text data is divided into paragraphs and sentences and stored to generate semi-structured text txt files; image files, audio and video files and video files are stored according to file names and address URLs to generate structured json files.

3. The method for constructing and adaptively visualizing a mobile intelligent graph according to claim 1, characterized in that: Step 2: Train the knowledge extraction model to perform entity recognition and relationship extraction on structured and semi-structured data to form a triple structure. Use entities as nodes and relationships as edges to build a mobile intelligent graph. The specific method is as follows: The knowledge extraction model uses the Qwen2-0.5b-Instruct pre-trained model to perform entity recognition and relationship extraction on structured and semi-structured data. The input is structured and semi-structured text data, and the output is entity recognition results and relationship extraction results. The Qwen2-0.5b-Instruct pre-trained model includes 4 dense models and 1 expert mixture model. The dense model parameter range includes 0.5B, 1.5B, 7B, and 72B, and the expert mixture model parameter range includes 57B-A14B; (2) Store the extracted entities in the database in the form of nodes to form an entity set: V s = {V1, V2, ... V n }, each entity V n It also contains a collection of attribute information: Organize the extracted relations in the form of edges, build the relations between the head entity and the tail entity, and form a relation set: E s ={E1, E2, ... E n }, each relation E n It also contains a collection of attribute information: For the constructed entity set V s and the relation set E s , create the entity table "vertex" and the relationship table "edge" in the mongodb database, and build the entity index and relational indexes Encapsulate the retrieval interface API to provide support for mobile-side intelligent graph visualization.

4. The method for constructing and adaptively visualizing a mobile intelligent graph according to claim 3, characterized in that: Entity Collection: V s = {V1, V2, ... V n }, each entity V n It also contains a collection of attribute information: in: The entity "Person" contains the attribute information "Chinese name", "English name", "Alias", "Nationality", "Gender", "Ethnicity", and "Date of Birth"; The entity "Organization" contains the attribute information "Chinese Name", "English Name", and "Introduction"; The entity "Party" contains the attribute information "Chinese name", "English name", and "Introduction"; The entity "place" contains the attribute information "Chinese name", "English name", "introduction", and "geographic coordinates"; The entity "Position" contains the attribute information "Chinese name", "English name", and "Job content"; The entity "school" contains the attribute information "Chinese name", "English name", and "introduction"; The entity "major" contains the attribute information "Chinese name", "English name", and "major course"; The entity "event" contains the attribute information "event location", "event content", "occurrence time", and "end time"; The entity "honor" contains attribute information "honor type" and "acquisition time"; The entity "achievement" contains attribute information "achievement content" and "acquisition time".

5. The method for constructing and adaptively visualizing a mobile intelligent graph according to claim 3, characterized in that: Relationship set: E s ={E1, E2, ... E n }, each relation E n It also contains a collection of attribute information: in: The relationship "Job Position" includes the head entity "Person" and the tail entity "Organization" and the attribute information "Position Held", "Job Start Time", and "Job End Time"; The relationship "participation" includes the head entity "person" and the tail entity "event" and the attribute information "participation time", "participation location", and "participation content"; The relationship "Political Party" includes the head entity "Person" and the tail entity "Party" and the attribute information "Participation Time" and "Participation Location"; The relationship "Graduated from" includes the head entity "Person" and the tail entity "Major" and the attribute information "Start Time", "Learning Place", "End Time", and "Degree Obtained"; The relation "education" includes the head entity "person" and the tail entity "school" and the attribute information "enrollment time", "education location", and "school leaving time"; The relationship "honor" includes the head entity "person" and the tail entity "honor" and the attribute information "honor type", "acquisition location", and "acquisition time"; The relationship "family affection" includes the head entity "person" and the tail entity "person" and the attribute information "husband and wife", "father and son", and "father and daughter"; The relationship "social" includes the head entity "person" and the tail entity "person" and the attribute information "friend" and "colleague"; The relationship "influence" includes the head entity "person" and the tail entity "achievement" and the attribute information "acquisition time" and "influence content"; The relationship "birthplace" includes the head entity "person" and the tail entity "place" and the attribute information "birthtime"; The relationship "residence" includes the head entity "person" and the tail entity "place" and the attribute information "residence time"; The relationship "other" includes the head entity "person" and the tail entity "event" and the attribute information "start time" and "other content".

6. The method for constructing and adaptively visualizing a mobile intelligent graph according to claim 1, characterized in that: Step 3: Get the relevant parameters of the mobile device, including screen width and height, screen brightness, and screen refresh frame rate. The specific method is as follows: (1) Screen width and height: Android devices use the DisplayMetrics class to obtain the width and height of the screen, initialize the class object DisplayMetrics displayMetrics = new DisplayMetrics(); then get the value of the object according to the getMetrics function getWindowManager().getDefaultDisplay().getMetrics(displayMetrics); finally get the width and height parameters int width = displayMetrics.widthPixels; int height = displayMetrics.heightPixels; IOS devices use the UIScreen class to get the width and height of the current screen, let screenWidth = UIScreen.main.bounds.size.width; let screenHeight = UIScreen.main.bounds.size.height; (2) Screen brightness: Android devices use the WindowManager class to obtain the current screen brightness, initialize the class object WindowManager wm = (WindowManager) context.getSystemService (Context.WINDOW_SERVICE); then use the getBrightness() function to obtain the screen brightness int brightness = wm.getDefaultDisplay().getBrightness(); IOS devices use the UIScreen class to get the current screen brightness, let screenBrightness = UIScreen.mainScreen().brightness; (3) Screen refresh frame rate refreshrate: Android devices use the DisplayMetrics class to obtain the refresh frame rate of the screen, initialize the class object DisplayMetrics displayMetrics = new DisplayMetrics(); then obtain the refresh frame rate through the getRefreshRate() function, getWindowManager().getDefaultDisplay().getRefreshRate(); The iOS device uses the UIScreen class to obtain the refresh frame rate of the current screen, then initializes the class object and obtains the refresh frame rate, let screen = UIScreen.mainScreen(); let refreshRate:Double = screen.nativeOrientation == .portrait? 60:

120.

7. The method for constructing and adaptively visualizing a mobile intelligent graph according to claim 1, characterized in that: Step 3: Obtain relevant parameters of the mobile device, including screen width and height, screen brightness, and screen refresh frame rate; establish a color template, predefine color RGB values ​​for entities and relationships, and form a visual color correspondence table for entities and relationships. The specific method is as follows: (1) The entities include the following: People, organizations, parties, places, positions, schools, majors, events, honors, and achievements; the color RGB value correspondence table is as follows: The character's color RGB value is set to (215, 135, 105); The color RGB value of the organization is set to (150, 125, 60); The RGB value of the party color is set to (105, 150, 60); The RGB value of the location color is set to (75, 150, 60); The color RGB value of the position is set to (35, 125, 95); The RGB value of the school color is set to (85, 75, 125); The professional color RGB value is set to (115, 75, 165); The event color RGB value is set to (125, 85, 95); The RGB value of the honor color is set to (215, 55, 55); The RGB value of the achievement color is set to (65, 25, 190); (2) The relationships are as follows: Position, Participation, Political Party, Graduated from, Education, Honor, Family, Social, Influence, Birthplace, Residence, Others; the color RGB value correspondence table is as follows: The RGB value of the color in use is set to (255, 15, 15); The RGB values ​​of the participating colors are set to (15, 255, 15); The RGB value of the political party color is set to (105, 255, 105); The RGB value of the graduated color is set to (255, 255, 0); The color RGB value of education is set to (185, 175, 125); The RGB value of the honor color is set to (55, 25, 85); The RGB value of the color of affection is set to (205, 25, 45); The RGB value of the social color is set to (205, 115, 85); The affected color RGB value is set to (85, 65, 45); The RGB value of the birthplace color is set to (205, 95, 95); The RGB value of the color of the residence is set to (90, 30, 30); The RGB values ​​of other colors are set to (120, 150, 160).

8. The method for constructing and adaptively visualizing a mobile intelligent graph according to claim 1, characterized in that: Step 4: Call the graph retrieval interface to obtain the number of relationship categories and the number of entity nodes under each relationship, dynamically calculate the layout according to the visualization template parameters, and draw the mobile terminal intelligent graph by classification aggregation and layered rendering, where: (1) The classification aggregation method is as follows: When the total number of knowledge graph data nodes is large, the relationship is used as the classification target, the entities of the same relationship are aggregated and rendered in the form of nodes. Due to the limited visualization space, the entities of the same relationship category are set with specified threshold parameters. If the number of Node count >3, it is displayed in an aggregated form; if the number of Node count If the value is less than 3, it will be directly rendered and displayed; the entity node name display and attribute field information display are expressed in an explicit and implicit way, that is, the name is the only visual field, and the attribute field information is displayed as the field displayed after interaction; (2) The layered rendering method is as follows: In order to solve the problem of rendering a large number of nodes at the same time in the visual interface, the node rendering method is adopted to display the total number of nodes that need to be rendered under the current relationship category. count , calculate the total number of layers that need to be rendered according to the classified aggregated nodes count , and then calculate the number of rendering nodes of the current layer according to the number of layers When the total number of nodes under each type of relationship count >57 and the total number of layers count >2, the total number of layers rendered is calculated using the formula: 40+5*(3+Layer count )*(Layer count -2)=Node count That is: First layer count＝1 hour, Second Layer count＝2 hour, The third layer count＝3 hour, The fourth layer count＝4 hour, ... Layer n count＝n hour, 9. The method for constructing and adaptively visualizing a mobile intelligent graph according to claim 1, characterized in that: Also includes: Step 5: Use Cytoscape.js graph rendering and interactive visualization tools for display, and support interactive actions including: touch, slide, zoom, classification aggregation, topological space extension, parameter feedback, among which: (1) The topological space is extended as follows: Topological space extension includes: dynamically allocating space according to the display results of different entity nodes; extending the display range space to the corresponding direction according to the relationship connection between nodes; and realizing the circular extension of nodes; (2) Parameter feedback is as follows: Parameter feedback includes the width and height of the mobile device screen, screen brightness, screen refresh rate, and the custom RGB value of the color template.

10. A mobile terminal intelligent graph construction and adaptive visualization system, characterized in that: Implement the mobile terminal intelligent graph construction and adaptive visualization method described in any one of claims 1-9 to realize mobile terminal intelligent graph construction and adaptive visualization.

Images