Multi-source heterogeneous data intelligent analysis method and system

Abstract

The invention relates to the technical field of large model security evaluation, in particular to a multi-source heterogeneous data intelligent analysis method and system. According to the method, firstly, multi-source heterogeneous data are comprehensively collected through a decompilation-free technology, a system obtains static attributes, dynamic interaction behaviors and associated sensitive content of a large model, and a complete evidence chain covering a front-end interface, interaction logic and rear-end data is constructed; then, realizing deep correlation analysis by constructing three special knowledge maps, and converting discrete multi-source heterogeneous data into a structured knowledge network; and then, based on information in the knowledge graph, risks such as over-range collection, data insecurity, control disguise and the like of the abstract large model are converted into computable and comparable numerical indexes, so that security assessment spans from subjective qualification to objective quantification, and a security assessment result of the large model is obtained.

Description

Technical Field

[0001] This invention relates to the field of large-scale model security assessment technology, specifically to a method and system for intelligent analysis of multi-source heterogeneous data. Background Technology

[0002] With the rapid development of generative artificial intelligence technology, deeply integrating large language models and multimodal models (collectively referred to as large models) into smart terminal applications in the form of independent functional controls (hereinafter referred to as "large model controls") has become a core technical path to promote the intelligent upgrading of services in various industries. These controls are usually embedded modules or interaction entry points within smart terminal applications, providing users with intelligent services such as dialogue, generation, analysis, and decision-making, significantly improving the naturalness and value density of application interactions.

[0003] Meanwhile, the unique operating mechanism of large-model controls brings security challenges that traditional applications have never faced. To improve service accuracy, controls may induce or implicitly collect excessive user contextual information (such as device information and irrelevant chat logs) during interactions, but insufficient encryption strength during transmission can lead to data leakage at multiple stages. More insidious are the ethical and fraud risks at the design level; controls may weaken users' ability to give informed consent by mimicking official UI styles, concealing risk warnings, or simplifying authorization processes, leading to the disclosure of critical sensitive information without full understanding. Therefore, conducting a specialized and in-depth security assessment of large-model controls built into smart terminal applications is of paramount importance.

[0004] However, the industry currently lacks mature and systematic assessment solutions in this area, and existing technologies are clearly insufficient. For example, mainstream smart terminal application security scanning tools focus on traditional levels such as code vulnerabilities, permission declarations, and network communication protocols. They typically mechanically check checklist-style permissions or data items, failing to effectively simulate and verify core interaction processes related to model controls, resulting in low accuracy in security detection.

[0005] In summary, in the face of the new security challenges brought about by the large models embedded in smart terminal applications in the form of controls, the industry urgently needs a data intelligence analysis method that can perform automated, in-depth, and measurable data analysis based on comprehensive multi-source heterogeneous data, so as to achieve accurate security assessment of large models. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for intelligent analysis of multi-source heterogeneous data, used to assess the security risks of large models existing in the form of controls in smart terminal applications.

[0007] The technical solution of this invention is as follows: A multi-source heterogeneous data intelligent analysis method is used to assess the security risks of large models existing in the form of controls in smart terminal applications, including the following operations: S1. Extract the control type, permission list, layout parameters, and text and / or image resource paths of the large model control to form basic attribute data; capture the control display duration, click-to-jump link, permission call timing, and cross-platform control rendering logs of the large model control to form dynamic behavior data; extract the ID card and / or bank card text, QR code data, and encryption status identifier associated with the large model control to obtain sensitive information data; S2. Based on the industry type, basic attribute data, dynamic behavior data and sensitive information data of the large model, construct a first knowledge graph with control type, permission type and industry type as nodes, a second knowledge graph with data type, encryption status and control type as nodes, and a third knowledge graph with control appearance characteristics, behavior characteristics and standard type as nodes. S3. In the first knowledge graph, the matching degree of control type nodes and permission type nodes is weighted and summed with the matching degree of industry type nodes and permission type nodes to obtain the risk value of out-of-range collection; the matching degree is the edge weight between the corresponding nodes; in the second knowledge graph, the matching degree of data type nodes and encryption state nodes is used as the data collection security risk value; in the third knowledge graph, the matching degree of appearance feature nodes and standard type nodes is weighted and summed with the matching degree of behavior feature nodes and standard type nodes to obtain the risk value of large model spoofing. S4. Based on the risk values ​​of out-of-range data acquisition, data acquisition security risk values, and control spoofing risk values, the security assessment results of the large model are obtained.

[0008] The method for obtaining sensitive information data in S1 is as follows: the original large model associated data is subjected to format standardization and adaptive enhancement processing to obtain standardized enhanced data; the standardized enhanced data is processed by the YOLOv8 model to obtain the region of interest; the region of interest is subjected to optical character recognition to extract text content and QR code data; the encryption header features in the standardized enhanced data are identified to obtain the encryption status identifier.

[0009] The operation of optical character recognition is as follows: the region of interest is denoised, histogram equalized and image enhanced, and edge enhancement is performed based on the Laplacian operator to obtain a standardized feature image; multi-scale feature maps of the standardized feature images are obtained, fused and processed by the detection head, and an adaptive threshold segmentation algorithm is used to generate a text region probability map, and pixel-level confidence is calculated to obtain the text region coordinates and confidence matrix; The text region coordinates and the standardized feature image are processed by region cropping and perspective transformation to obtain an orthophoto text image; the orthophoto text image is processed by CNN feature extraction, Transformer encoding and CTC decoding to obtain the text content; the standardized feature image and the non-text region coordinates are processed by QR code positioning and multi-protocol decoding to obtain QR code data.

[0010] In S3, the matching degree between control type nodes and permission type nodes is based on the proportion of call frequency, dependency strength, concentration of call timing, and encryption correlation; the matching degree between industry type nodes and permission type nodes is based on compliance correlation, industry usage coverage, and risk matching degree.

[0011] The matching degree between data type nodes and encrypted state nodes in S3 is obtained based on encryption coverage, sensitivity drive, decryption success rate, and industry default.

[0012] In S3, the matching degree between appearance feature nodes and standard type nodes is obtained based on visual matching degree, resource matching degree, identifier matching degree, and industry adaptability; the matching degree between behavior feature nodes and standard type nodes is obtained based on behavior conformity, call adaptability, stability adaptability, and scene adaptability.

[0013] The large-scale model security assessment results in S4 include the comprehensive security risk value of the large-scale model, calculated using the following formula: , , The comprehensive safety risk value of the large model. This is to identify risk values ​​collected beyond the permitted scope, data collection security risks, or large models used to disguise risk values. i =1,2,3 For entropy weighting coefficients, To address the risks associated with collecting data beyond its permitted scope, or the risks associated with data acquisition or control spoofing, As the industry's urgency factor, The maximum value is the weighted fusion of historical out-of-range collection risk values, data collection security risk values, and large model spoofing risk values.

[0014] A multi-source heterogeneous data intelligent analysis system, used to implement the above-mentioned multi-source heterogeneous data intelligent analysis method, includes: The multi-source heterogeneous data acquisition module is used to extract control types, permission lists, layout parameters, and text and / or image resource paths of large model controls without decompilation, forming basic attribute data; it captures the display duration of large model controls, the redirection link after clicking, the timing of permission calls, and cross-platform control rendering logs, forming dynamic behavior data; and it extracts the ID card and / or bank card text, QR code data, and encryption status identifier associated with large model controls to obtain sensitive information data. The knowledge graph generation module is used to construct a first knowledge graph with control type, permission type, and industry type as nodes, a second knowledge graph with data type, encryption status, and control type as nodes, and a third knowledge graph with control appearance features, behavior features, and standard type as nodes, based on the industry type, basic attribute data, dynamic behavior data, and sensitive information data of the large model. The multi-dimensional risk value generation module is used to perform a weighted summation of the matching degree between control type nodes and permission type nodes and the matching degree between industry type nodes and permission type nodes in the first knowledge graph to obtain the out-of-range collection risk value; the matching degree is the edge weight between the corresponding nodes; the matching degree between data type nodes and encryption state nodes in the second knowledge graph is used as the data collection security risk value; the matching degree between appearance feature nodes and standard type nodes and the matching degree between behavior feature nodes and standard type nodes in the third knowledge graph is performed a weighted summation to obtain the large model spoofing risk value. The large model security assessment result generation module is used to obtain the large model security assessment result based on the risk value of out-of-range acquisition, the data acquisition security risk value, and the control spoofing risk value.

[0015] A multi-source heterogeneous data intelligent analysis device includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the above-mentioned multi-source heterogeneous data intelligent analysis method.

[0016] A computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above-described intelligent analysis method for multi-source heterogeneous data.

[0017] The beneficial effects of this invention are as follows: This invention provides an intelligent analysis method for multi-source heterogeneous data, used for security assessment of large models existing in the form of controls in smart terminal applications. The method first comprehensively collects multi-source heterogeneous data using decompilation-free technology. The system acquires the static attributes, dynamic interactive behaviors, and associated sensitive content of the large model, constructing a complete evidence chain covering the front-end interface, interaction logic, and back-end data, laying a comprehensive and objective factual foundation for in-depth security analysis. Then, it achieves deep correlation analysis by constructing three specialized knowledge graphs, transforming discrete multi-source heterogeneous data into a structured knowledge network. This not only facilitates accurate revelation of compliance contradictions between large model permissions and industry standards, and security gaps between sensitive data and protection status, but also identifies deceptive differences between interface design and true intent, achieving in-depth deconstruction of complex risks. Next, based on the information in the knowledge graphs, abstract risks such as "excessive data collection," "data insecurity," and "control spoofing" of the large model are transformed into calculable and comparable numerical indicators, enabling security assessment to move from subjective qualitative to objective quantitative, obtaining the security assessment results of the large model. The present invention provides a multi-source heterogeneous data intelligent analysis method. From multi-source data collection and knowledge graph construction to risk value calculation and comprehensive assessment, the entire process does not require deep human intervention. This not only greatly improves the efficiency and consistency of assessment, but also provides clear and operable decision-making basis for subsequent security rectification, compliance audit and continuous monitoring with its output quantitative results. This invention provides a multi-source heterogeneous data intelligent analysis method that starts from the front-end interactive interface of smart terminal applications and goes directly to the back-end data of large models. Through multi-dimensional graph correlation, the system assesses the security risks of large models, surpassing traditional vulnerability scanning. It can discover hidden problems that are "superficially compliant but actually overstepping boundaries", and realizes automated, intelligent and accurate assessment of the deep security risks of large models built into smart terminal applications. Attached Figure Description

[0018] The solutions and advantages of this application will become clear to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of the invention.

[0019] In the attached diagram: Figure 1 This is a flowchart illustrating the method of this embodiment. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the exemplary embodiments of this application clearer, the technical solutions in the exemplary embodiments of this application are described clearly and completely below. Obviously, the described exemplary embodiments are only some embodiments of this application, and not all embodiments.

[0021] Based on the exemplary embodiments shown in this application, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this application. Furthermore, although the disclosures in this application are presented by way of one or more exemplary examples, it should be understood that each aspect of these disclosures can constitute a complete technical solution on its own.

[0022] Furthermore, the terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclusively include, for example, a product or device that includes a series of components is not necessarily limited to those that are explicitly listed, but may include other components that are not explicitly listed or that are inherent to such product or device.

[0023] As used in this application, the term "module" means any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code capable of performing the functions associated with that element.

[0024] Example 1 This embodiment provides a multi-source heterogeneous data intelligent analysis method for assessing the security risks of large models existing in the form of controls in smart terminal applications, including the following operations: S1. Extract the control type, permission list, layout parameters, and text and / or image resource paths of the large model control to form basic attribute data; capture the control display duration, click-to-jump link, permission call timing, and cross-platform control rendering logs of the large model control to form dynamic behavior data; extract the ID card and / or bank card text, QR code data, and encryption status identifier associated with the large model control to obtain sensitive information data; S2. Based on the industry type, basic attribute data, dynamic behavior data and sensitive information data of the large model, construct a first knowledge graph with control type, permission type and industry type as nodes, a second knowledge graph with data type, encryption status and control type as nodes, and a third knowledge graph with control appearance characteristics, behavior characteristics and standard type as nodes. S3. In the first knowledge graph, the matching degree of control type nodes and permission type nodes is weighted and summed with the matching degree of industry type nodes and permission type nodes to obtain the risk value of out-of-range collection; the matching degree is the edge weight between the corresponding nodes; in the second knowledge graph, the matching degree of data type nodes and encryption state nodes is used as the data collection security risk value; in the third knowledge graph, the matching degree of appearance feature nodes and standard type nodes is weighted and summed with the matching degree of behavior feature nodes and standard type nodes to obtain the risk value of large model spoofing. S4. Based on the risk values ​​of out-of-range data acquisition, data acquisition security risk values, and control spoofing risk values, the security assessment results of the large model are obtained.

[0025] The specific steps are detailed below.

[0026] S1. Extract the control type, permission list, layout parameters, and text and / or image resource paths of the large model control to form basic attribute data; capture the control display duration, click-and-jump link, permission call timing, and cross-platform control rendering logs of the large model control to form dynamic behavior data; extract the ID card and / or bank card text, QR code data, and encryption status identifier associated with the large model control to obtain sensitive information data.

[0027] By acquiring large-scale models existing in the form of controls in smart terminal applications, we can define the basic framework for risk analysis and the static basic attribute data of compliance benchmarks. We can also accurately record the triggering path and dynamic behavior data of risks during the interaction process, as well as the core hazard evidence and sensitive information data. This has built a complete and comprehensive multi-source heterogeneous data foundation, forming a data system that is "from surface to core, combining static and dynamic data". This allows subsequent security assessments to transcend the limitations of traditional single dimensions and achieve an objective analysis of the entire chain of large-scale model controls, from interface design and interaction logic to data leakage. It fully describes the entire lifecycle of the large-scale model from "what it is" to "what it does" and then to "what data it touches", providing accurate and traceable factual evidence for subsequent quantitative risk assessment.

[0028] This method extracts large-scale model control types (e.g., buttons, input boxes, QR code display controls), permission lists (e.g., read storage permissions, camera permissions, location permissions, payment permissions), layout parameters (including position and size), and text and / or image resource paths without decompilation. This provides foundational data for out-of-scope data collection detection and control spoofing detection, forming basic attribute data. Decompilation-free extraction can be achieved through Android AccessibilityService / iOS Accessibility API + Frida memory scanning, shifting from the source code layer to the control layer. This not only avoids the problems of traditional decompilation, such as protection triggering, difficult adaptation, and high costs, but also ensures that the obtained data originates from the actual running state of the application and user interactions, rather than simple trust in developer claims or indirect speculation from the source code, greatly improving the objectivity and evidentiary value of the data.

[0029] Meanwhile, through Taiji's rootless instrumentation, the display duration of large model controls, the redirection path after clicking, the timing of permission calls (such as calling the address book permission when triggering a large model control), and cross-platform control rendering logs (such as FlutterWidget loading logs) can be collected. This provides behavioral evidence for out-of-scope collection and detection, forming dynamic behavioral data that reflects how risks are triggered and transmitted in actual interactions.

[0030] Furthermore, by extracting the ID card and / or bank card text, QR code data, and encryption status identifier associated with the large model control, sensitive information data is obtained, directly pointing to core evidence of harm such as privacy leaks and data security breaches.

[0031] The specific steps for obtaining sensitive information data are as follows.

[0032] Step 1: Standardize the format of the original large model associated data (which can be obtained by capturing packets through network traffic analysis), unify the data storage format, resolution, and encoding method, and then optimize the data quality through adaptive enhancement (including brightness adjustment, noise removal, and distortion correction) to obtain standardized enhanced data.

[0033] Step 2: The standardized augmented data is processed by the YOLOv8 model to accurately locate the text area and QR code area of ​​the ID card or / and bank card, and obtain the region of interest; the region of interest is then processed by optical character recognition to extract the text content and QR code data; at the same time, the encryption header features in the standardized augmented data are identified by the byte feature matching method to obtain the encryption status identifier.

[0034] The optical character recognition process is as follows: The region of interest (ROI) undergoes denoising, histogram equalization, and image enhancement (achieved via the Retinex enhancement algorithm). Brightness is then calibrated for low-light or overexposed images, and edge enhancement is performed using the Laplacian operator to obtain a standardized feature image. Multi-scale feature maps of the standardized feature image are acquired (based on a CSPNet network). After fusing the multi-scale feature maps via feature pyramids, the detection head processes the data and uses an adaptive threshold segmentation algorithm to generate a text region probability map. Pixel-level confidence is calculated to obtain the text region coordinates and confidence matrix. The text region coordinates and the standardized feature image undergo region cropping and perspective transformation to obtain an orthorectified text image. The orthorectified text image is then processed by CNN feature extraction and Transformer encoding to extract local texture features and capture long-distance character dependencies. CTC decoding, combined with a custom character dictionary (containing numbers, letters, common symbols, and character sets specific to ID cards and bank cards), is used to decode the optimal text sequence using a beam search algorithm, resulting in structured text content. The standardized feature image and non-text region coordinates are then processed by QR code localization and multi-protocol decoding to obtain QR code data.

[0035] Step 3: Link and integrate the text content, QR code data, and encryption status identifier, remove duplicate and invalid data, verify the text format (such as the 18-digit ID card code and bank card number check digit), the integrity of the QR code data, and the accuracy of the encryption identifier, and finally form structured sensitive information data.

[0036] S2. Based on the industry type, basic attribute data, dynamic behavior data, and sensitive information data of the large model, construct a first knowledge graph with control type, permission type, and industry type as nodes, a second knowledge graph with data type, encryption status, and control type as nodes, and a third knowledge graph with control appearance characteristics, behavior characteristics, and standard type as nodes.

[0037] By constructing three specialized knowledge graphs, complex data is transformed into a quantifiable and assessable logical framework. The first graph (controls-permissions-industry) logically reveals whether permission requests exceed the necessary business scope by comparing the permissions required for control functions with industry compliance benchmarks, supporting the quantitative calculation of risks associated with data collection beyond the scope. The second graph (data-encryption-controls) forcibly associates sensitive data types with their actual protection status, establishing a core logical link for assessing the security of data collection. The third graph (appearance-behavior-standard type) systematically deconstructs the inherent logic of inducement and disguise by analyzing the contradictory relationship between the surface features and underlying behaviors of the large model, so that the risk of disguise no longer depends on subjective judgment, but is objectively quantified based on measurable feature mismatch.

[0038] The construction of the first knowledge graph.

[0039] By constructing a structure graph with control type, permission type, and industry type as nodes and the relationships between nodes as edges, we can integrate the basic attributes, dynamic behaviors, and sensitive information data of controls to form a structured association, thus obtaining the first knowledge graph. This graph is used to solve the compliance problem of "why this permission is needed," which is the key to evaluating out-of-scope data collection.

[0040] The node characteristics of control type nodes are formed by the control type identifier, text and / or image resource paths from the basic attribute characteristics of the control, the average display duration of the control, the depth of the jump link after clicking, and cross-platform rendering compatibility (obtained from cross-platform control rendering logs) from the dynamic behavior characteristics, and sensitive association characteristics. The node characteristics of control type nodes not only clarify the core identity and associated resources of the control through the control type identifier and resource paths, but also restore the actual running scenario and usage status of the control through dynamic behavior characteristics such as average display duration and jump link depth. Furthermore, the sensitive association characteristics accurately identify whether the control involves sensitive information such as ID cards and bank cards, as well as encrypted attributes, achieving multi-dimensional and comprehensive coverage of control attributes, behaviors, and sensitive associations.

[0041] Among them, sensitive association features include whether it is associated with ID card and / or bank card text, whether it is associated with QR code data, and whether it contains encrypted status identifiers.

[0042] The node features of permission types consist of large model functional features that clearly define the core purpose of the permission, call features that facilitate the identification of the appropriate timing for permission use (call timing, including startup / after clicking / running in the background), risk features that quantify the association of permissions with sensitive information (sensitive information data, such as associated ID cards or bank cards), and permission dependency features that can restore the historical call patterns and relationships of permissions (a list of historically called permission types and the percentage of permission call frequencies). These multi-dimensional features work together to form a three-dimensional understanding of permissions, effectively avoiding misjudgments caused by excessive data collection due to a single feature dimension.

[0043] The edges between control type nodes and permission type nodes are established based on the control's permission list, the timing of permission calls in dynamic behavior data, and sensitive information data. They reflect the necessary or optional permission associations for the control types in the large model to achieve their functions. It is important to focus on the necessary or optional permission associations for the large model to achieve its functions and ensure that the establishment of the edges has a clear business scenario and data.

[0044] The edge features between control type nodes and permission type nodes are obtained from the following: call frequency ratio (the ratio of the number of times the control calls the corresponding permission to the total number of permissions called by the control), dependency strength (whether the permission is necessary for the core function of the control, strong dependency = 1, weak dependency = 0.3, no dependency = 0), call timing concentration (the degree to which permission calls are concentrated at startup or at specific times after clicking), and encryption association (the degree of association between permission calls and encryption status indicators, i.e., the percentage of time the permission is used when calling encrypted information).

[0045] The edge weight (matching degree) between control type nodes and permission type nodes is a weighted sum of frequency ratio, dependency strength, call timing concentration, and encryption association degree.

[0046] The edge features of industry type nodes and permission type nodes are established based on the compliance requirements of industry type, dynamic behavior data of controls, and sensitive information data, reflecting the high-frequency or mandatory use of a certain permission type in a certain industry scenario.

[0047] The edge features of industry type nodes and permission type nodes are derived based on compliance relevance (whether the permission is mandatory according to industry compliance standards; mandatory requirement = 1, recommended requirement = 0.7, no requirement = 0), industry usage coverage (the ratio of the number of controls using the permission type within the industry to the total number of controls within the industry), and risk matching degree (the degree to which the permission risk level matches the industry's sensitive information protection requirements). The edge weights (matching degrees) of industry type nodes and permission type nodes are the weighted sum of compliance relevance, industry usage coverage, and risk matching degree.

[0048] The construction of the second knowledge graph.

[0049] Constructing a second knowledge graph with data types (e.g., ID card text, bank card text, QR code data, plain text, plain image data), encryption status (e.g., AES encrypted header, no encryption), and control types as nodes can intuitively display the protection status of data during transmission or storage, and is the core of assessing the security risks of data collection.

[0050] The edges between data type nodes and encryption state nodes are constructed based on sensitive information data (the binding relationship between data type and encryption state) and industry type (industry encryption standards), reflecting the association that a certain data type is often in a certain encryption state.

[0051] The edge features of data type nodes and encryption state nodes are obtained based on encryption coverage (the ratio of the number of data types in this encryption state to the total number of data types), sensitivity drive (the product of the data type sensitivity level and the encryption ratio of the corresponding sensitivity level data), decryption success rate (the ratio of the number of times the data type in the encryption state is successfully decrypted to the total number of decryptions), and industry default (the proportion of data types in the industry that default to this encryption state). The edge weight (matching degree) of data type nodes and encryption state nodes is the weighted sum of encryption coverage, sensitivity drive, decryption success rate, and industry default.

[0052] The construction of the third knowledge graph.

[0053] The third knowledge graph is constructed with nodes based on control appearance features (e.g., rectangular input box appearance, circular button appearance, QR code pattern display appearance, ID card card display appearance), behavioral features (e.g., short-duration display behavior, click-to-jump multi-link behavior, high-frequency background camera call behavior, cross-platform adaptation anomaly behavior), and standard types (e.g., ID card entry control, bank card input control, QR code display control, ordinary login button, payment confirmation control). It targets the "obfuscation" or "fraud" risks unique to large model scenarios. For example, a seemingly ordinary input box may actually send user input to a large model for sensitive analysis in the background.

[0054] The edges between appearance feature nodes and standard type nodes are established based on basic attribute data (layout parameters, text / image resource paths) and the baseline appearance parameters of standard type nodes, reflecting the correlation between whether the appearance features of the large model conform to the standard appearance of the standard type.

[0055] The edge features between appearance feature nodes and standard type nodes are obtained based on visual matching degree (the overlap score between the shape / size / color of the appearance feature and the appearance of the standard type), resource matching degree (the consistency score between the text / image resources associated with the appearance and the resources of the standard type), identifier matching degree (the degree of conformity between the appearance-specific identifiers (such as cursors, locators) and the identifiers of the standard type), and industry adaptability (the commonness of the appearance type in its industry). The edge weight (matching degree) between appearance feature nodes and standard type nodes is the weighted sum of visual matching degree, resource matching degree, identifier matching degree, and industry adaptability degree.

[0056] The edges between behavioral feature nodes and standard type nodes are established based on dynamic behavioral data (control display duration, jump path, permission call timing) and standard behaviors of standard type nodes, reflecting whether the behavioral feature matches a certain standard type behavior.

[0057] The edge features of behavioral feature nodes and standard type nodes are obtained based on behavioral fit (the matching score between the display duration / jump link / call timing of the behavioral feature and the standard type behavior), call adaptability (the degree of matching between the permission call type in the behavior and the permission list of the standard type), stability adaptability (the degree of fit between cross-platform rendering stability, behavior response stability and standard type stability), and scene adaptability (the matching score between the behavior occurrence scene and the standard type functional scene). The edge weight (matching degree) is the weighted sum of behavioral fit, call adaptability, stability adaptability and scene adaptability.

[0058] S3. In the first knowledge graph, the matching degree of control type nodes and permission type nodes is weighted and summed with the matching degree of industry type nodes and permission type nodes to obtain the out-of-range collection risk value; in the second knowledge graph, the matching degree of data type nodes and encryption status nodes is used as the data collection security risk value; in the third knowledge graph, the matching degree of appearance feature nodes and standard type nodes is weighted and summed with the matching degree of behavior feature nodes and standard type nodes to obtain the large model spoofing risk value.

[0059] By transforming the complex relationships in knowledge graphs into calculable and comparable numerical indicators, this approach represents a crucial leap from qualitative description to quantitative measurement in security assessment. Specifically, by calculating the weighted sum of edge weights between controls, permissions, and industries, the vague concept of out-of-scope data collection is transformed into precise numerical values ​​based on business scenarios, thus standardizing compliance judgments. Furthermore, by directly mapping the matching degree between data types and encryption states, a direct quantitative benchmark for data security risks is established, clearly revealing the gap between protection measures and data sensitivity. Finally, by comprehensively assessing the mismatch between appearance and behavior features, hidden deception risks are transformed into measurable anomaly scores. This entire methodology not only unifies the standards for risk assessment and enables cross-application and cross-risk prioritization, but also, through structured computational logic, allows for the tracing of risk causes, providing direct data-driven decision-making support for precise and efficient security hardening.

[0060] The risk value collected beyond the scope covers two core risk dimensions: "control-permission matching" and "industry-permission compliance". The calculation formula is as follows: , , To collect risk values ​​beyond the permitted range, , The matching degree is calculated for control type nodes and permission type nodes, and for industry type nodes and permission type nodes. This considers both the rationality of the permission requirements of the large model controls themselves and industry compliance constraints, avoiding misjudgment based on a single dimension. , These are the weights for large-scale model requirements and industry compliance. For industry compliance matching degree entropy, large model controls often involve cross-industry permission calls. Matching degree entropy can amplify the risk of matching degree being in an intermediate state, solving the problem that traditional formulas are insufficient for identifying risks in ambiguous compliance scenarios. , These are the industry sensitivity coefficient and the access risk level coefficient, respectively. As the first compensation amount, This is the coefficient for exceptions related to authorized access.

[0061] The formula for calculating the data acquisition security risk value is as follows: , , For data collection security risk values, To determine the matching degree between data type nodes and encrypted state nodes, through... The matching degree between data type and encryption state is converted into risk contribution. When highly sensitive data collected by the large model (such as user command text) does not match the encryption state, this factor will non-linearly amplify the risk, avoiding the underestimation of risk in low-match, high-risk scenarios by traditional linear formulas. To ensure data encryption compliance and matching degree entropy, , These are the data sensitivity coefficient and the encryption strength coefficient, respectively. To dynamically collect anomaly coefficients, This is the second compensation amount.

[0062] The formula for calculating the risk value of large model camouflage is as follows: , , To disguise the risk value for a large model, , These are the matching degrees between appearance feature nodes and standard type nodes, and the matching degrees between behavior feature nodes and standard type nodes, respectively. When a large model is disguised as a regular button... The value will be too low, when a large model disguises itself as a regular button but calls high-level permissions. The value will be too low, through , Low matching degree is directly converted into risk contribution. The higher the degree of spoofing of a large model, the lower the matching degree and the higher the risk value. , These are appearance weight and behavior weight, respectively. , These are appearance matching entropy and behavior matching entropy, respectively. The abnormality coefficient is the coefficient of behavior. This is the type confidence coefficient.

[0063] S4. Based on the risk values ​​of out-of-range data collection, data collection security risk values, and large model spoofing risk values, the security assessment results of the large model are obtained.

[0064] If the risk values ​​for collecting data beyond the permitted scope, the data collection security risk values, and the risk values ​​for spoofing large models exceed their respective thresholds, then corresponding risk warnings will be issued.

[0065] Furthermore, based on the risk values ​​of out-of-range data collection, data collection security risk values, and large model spoofing risk values, the comprehensive security risk value of the large model is calculated, and the security risk level of the large model is obtained by matching.

[0066] The formula for calculating the comprehensive safety risk value of the large model is as follows: , , The comprehensive safety risk value of the large model. This is to identify risk values ​​collected beyond the permitted scope, data collection security risks, or large models used to disguise risk values. i =1,2,3 For entropy weighting coefficients, The risk coupling degree for out-of-range data acquisition, or the risk coupling degree for data acquisition security, or the risk coupling degree for control spoofing, is calculated based on the Pearson correlation coefficient of the risk value for out-of-range data acquisition, or the risk value for data acquisition security, or the risk value for large model spoofing. As the industry's urgency factor, The maximum value obtained by weighting and fusing the three types of historical risk values ​​(over-range collection risk value, data collection security risk value, and large model spoofing risk value) is used for normalization output.

[0067] The large model security risk level, along with the risk values ​​for out-of-range data collection, data collection security risks, and large model camouflage risks, form the large model security assessment results.

[0068] To facilitate understanding of the above technical solution, an example is given to illustrate the implementation logic and effect of the method in this embodiment. Taking a "smart health consultation smart terminal application" as an example, it has a seemingly ordinary "symptom description" text box (control), which is connected to a large model behind it. Following this solution step by step: S1 of this embodiment extracts its basic attributes (such as text box type, no direct high-risk permissions), dynamic behavior (after discovering user input of symptoms, the smart terminal application calls network permissions to send data to the cloud-based large model API), and potentially associated sensitive information (the input content may include medical history, etc.); when constructing the graph in S2, the first graph will find a "text box" control in the "health and medical" industry, whose behavior is associated with "network" permissions, used to upload highly sensitive health data. The first graph constitutes a suspicious "node relationship"; the second graph checks whether this health data is encrypted (e.g., whether the transmission uses HTTPS); the third graph analyzes whether this "symptom description box" is a "large model query interface" in behavior and substance, and not just a text box recorded locally, thereby identifying its "standard type"; S3 finally calculates high-risk values: risk of excessive collection (excessive uploading of medical data), data security risk (extremely high risk if the transmission is not encrypted), and risk of large model spoofing (users may be unaware of sharing privacy with cloud AI); this accurately detects the two core risks of "privacy leakage" and "lack of informed consent".

[0069] This embodiment also provides a multi-source heterogeneous data intelligent analysis system for implementing the above-mentioned multi-source heterogeneous data intelligent analysis method, including: The multi-source heterogeneous data acquisition module is used to extract control types, permission lists, layout parameters, and text and / or image resource paths of large model controls without decompilation, forming basic attribute data; it captures the display duration of large model controls, the redirection link after clicking, the timing of permission calls, and cross-platform control rendering logs, forming dynamic behavior data; and it extracts the ID card and / or bank card text, QR code data, and encryption status identifier associated with large model controls to obtain sensitive information data. The knowledge graph generation module is used to construct a first knowledge graph with control type, permission type, and industry type as nodes, a second knowledge graph with data type, encryption status, and control type as nodes, and a third knowledge graph with control appearance features, behavior features, and standard type as nodes, based on the industry type, basic attribute data, dynamic behavior data, and sensitive information data of the large model. The multi-dimensional risk value generation module is used to perform a weighted summation of the matching degree between control type nodes and permission type nodes and the matching degree between industry type nodes and permission type nodes in the first knowledge graph to obtain the out-of-range collection risk value; the matching degree is the edge weight between the corresponding nodes; the matching degree between data type nodes and encryption state nodes in the second knowledge graph is used as the data collection security risk value; the matching degree between appearance feature nodes and standard type nodes and the matching degree between behavior feature nodes and standard type nodes in the third knowledge graph is performed a weighted summation to obtain the large model spoofing risk value. The large model security assessment result generation module is used to obtain the large model security assessment result based on the risk value of out-of-range acquisition, the data acquisition security risk value, and the control spoofing risk value.

[0070] This embodiment also provides a multi-source heterogeneous data intelligent analysis device, including a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the above-described multi-source heterogeneous data intelligent analysis method.

[0071] This embodiment also provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above-described intelligent analysis method for multi-source heterogeneous data.

[0072] This embodiment provides a multi-source heterogeneous data intelligent analysis method for security assessment of large models existing in the form of controls in smart terminal applications. The method first comprehensively collects multi-source heterogeneous data using decompilation-free technology. The system acquires the static attributes, dynamic interactive behaviors, and associated sensitive content of the large model, constructing a complete evidence chain covering the front-end interface, interaction logic, and back-end data, laying a comprehensive and objective factual foundation for in-depth security analysis. Then, it achieves deep correlation analysis by constructing three specialized knowledge graphs, transforming discrete multi-source heterogeneous data into a structured knowledge network. This not only facilitates accurate revelation of compliance contradictions between large model permissions and industry standards, and security gaps between sensitive data and protection status, but also identifies deceptive differences between interface design and true intent, achieving in-depth deconstruction of complex risks. Next, based on the information in the knowledge graphs, abstract risks such as "excessive data collection," "data insecurity," and "control masquerading" of the large model are transformed into calculable and comparable numerical indicators, enabling the security assessment to move from subjective qualitative to objective quantitative, obtaining the security assessment results of the large model.

[0073] This embodiment provides a multi-source heterogeneous data intelligent analysis method. From multi-source data collection and knowledge graph construction to risk value calculation and comprehensive assessment, the entire process does not require deep human intervention. This not only greatly improves the efficiency and consistency of the assessment, but also provides clear and operable decision-making basis for subsequent security rectification, compliance audit and continuous monitoring with its output quantitative results.

[0074] This embodiment provides a multi-source heterogeneous data intelligent analysis method that starts from the front-end interactive interface of smart terminal applications and goes directly to the back-end data of large models. Through multi-dimensional graph correlation, the system assesses the security risks of large models, surpassing traditional vulnerability scanning. It can discover hidden problems that are "superficially compliant but actually overstepping boundaries", and realizes automated, intelligent and accurate assessment of the deep security risks of large models built into smart terminal applications.

[0075] While exemplary embodiments of the invention have been described herein, many other variations or modifications conforming to the principles of the invention can be directly determined or derived from the disclosure of this invention without departing from its spirit and scope. Therefore, the scope of the invention should be understood and recognized to cover all such other variations or modifications.

Claims

1. A multi-source heterogeneous data intelligent analysis method for assessing the security risks of large models existing in the form of controls in smart terminal applications, characterized in that... This includes the following operations: S1. Extract the control type, permission list, layout parameters, and text and / or image resource paths of the large model controls to form basic attribute data; By capturing the display duration of large model controls, the redirection path after clicking, the timing of permission calls, and cross-platform control rendering logs, dynamic behavioral data is formed. Extract the ID card and / or bank card text, QR code data, and encryption status identifier associated with the large model control to obtain sensitive information data; S2. Based on the industry type, basic attribute data, dynamic behavior data and sensitive information data of the large model, construct a first knowledge graph with control type, permission type and industry type as nodes, a second knowledge graph with data type, encryption status and control type as nodes, and a third knowledge graph with control appearance characteristics, behavior characteristics and standard type as nodes. S3. The matching degree of control type nodes and permission type nodes in the first knowledge graph is weighted and summed with the matching degree of industry type nodes and permission type nodes to obtain the risk value of out-of-range collection. The matching degree is the edge weight between the corresponding nodes; The matching degree between data type nodes and encrypted state nodes in the second knowledge graph is used as the data collection security risk value. The matching degree between appearance feature nodes and standard type nodes in the third knowledge graph is weighted and summed with the matching degree between behavior feature nodes and standard type nodes to obtain the large model spoofing risk value. S4. Based on the risk values ​​of out-of-range data acquisition, data acquisition security risk values, and control spoofing risk values, the security assessment results of the large model are obtained.

2. The intelligent analysis method for multi-source heterogeneous data according to claim 1, characterized in that, In S1, the method for obtaining sensitive information data is as follows: The original large model associated data is subjected to format standardization and adaptive augmentation to obtain standardized augmented data; The standardized augmented data is processed by the YOLOv8 model to obtain the region of interest; the region of interest is then processed by optical character recognition to extract the text content and QR code data; and the encryption header features in the standardized augmented data are identified to obtain the encryption status identifier.

3. The intelligent analysis method for multi-source heterogeneous data according to claim 2, characterized in that, The operation of optical character recognition is as follows: The region of interest is denoised, histogram equalized, and image enhanced. Edge enhancement is performed based on the Laplacian operator to obtain a standardized feature image. Multi-scale feature maps of the standardized feature images are obtained, fused, and then processed by the detection head. An adaptive threshold segmentation algorithm is used to generate a text region probability map. Pixel-level confidence is calculated to obtain the text region coordinates and confidence matrix. The text region coordinates and standardized feature images are processed by region cropping and perspective transformation to obtain an orthophoto text image; the orthophoto text image is then processed by CNN feature extraction, Transformer encoding and CTC decoding to obtain the text content; The standardized feature image and non-text region coordinates are processed through QR code localization and multi-protocol decoding to obtain QR code data.

4. The intelligent analysis method for multi-source heterogeneous data according to claim 1, characterized in that, In S3, the matching degree between control type nodes and permission type nodes is obtained based on the proportion of call frequency, dependency strength, concentration of call timing, and encryption association degree. The matching degree between industry type nodes and permission type nodes is obtained based on compliance relevance, industry usage coverage, and risk matching degree.

5. The intelligent analysis method for multi-source heterogeneous data according to claim 1, characterized in that, In S3, the matching degree between data type nodes and encrypted state nodes is obtained based on encryption coverage, sensitivity drive, decryption success rate, and industry default.

6. The intelligent analysis method for multi-source heterogeneous data according to claim 1, characterized in that, In S3, the matching degree between appearance feature nodes and standard type nodes is obtained based on visual matching degree, resource matching degree, identifier matching degree, and industry adaptability. The matching degree between behavioral feature nodes and standard type nodes is obtained based on behavioral consistency, invocation adaptability, stability adaptability, and scenario adaptability.

7. The intelligent analysis method for multi-source heterogeneous data according to claim 1, characterized in that, The large-scale model security assessment results in S4 include the comprehensive security risk value of the large-scale model, calculated using the following formula: ， ， The comprehensive safety risk value of the large model. This is to identify risk values ​​collected beyond the permitted scope, data collection security risks, or large models used to disguise risk values. i =1,2,3 For entropy weighting coefficients, To address the risks associated with collecting data beyond its permitted scope, or the risks associated with data acquisition or control spoofing, As the industry's urgency factor, It is the maximum value obtained by weighting and fusing historical out-of-range collection risk values, data collection security risk values, and large model spoofing risk values.

8. A multi-source heterogeneous data intelligent analysis system, used to implement the multi-source heterogeneous data intelligent analysis method of claim 1, characterized in that, include: The multi-source heterogeneous data acquisition module is used to extract control types, permission lists, layout parameters, and text and / or image resource paths of large model controls without decompilation, forming basic attribute data; it captures the display duration of large model controls, the redirection link after clicking, the timing of permission calls, and cross-platform control rendering logs, forming dynamic behavior data; and it extracts the ID card and / or bank card text, QR code data, and encryption status identifier associated with large model controls to obtain sensitive information data. The knowledge graph generation module is used to construct a first knowledge graph with control type, permission type, and industry type as nodes, a second knowledge graph with data type, encryption status, and control type as nodes, and a third knowledge graph with control appearance features, behavior features, and standard type as nodes, based on the industry type, basic attribute data, dynamic behavior data, and sensitive information data of the large model. The multi-dimensional risk value generation module is used to perform a weighted summation of the matching degree of control type nodes and permission type nodes with the matching degree of industry type nodes and permission type nodes in the first knowledge graph to obtain the risk value of out-of-range collection. The matching degree is the edge weight between the corresponding nodes; The matching degree between data type nodes and encrypted state nodes in the second knowledge graph is used as the data collection security risk value. The matching degree between appearance feature nodes and standard type nodes in the third knowledge graph is weighted and summed with the matching degree between behavior feature nodes and standard type nodes to obtain the large model spoofing risk value. The large model security assessment result generation module is used to obtain the large model security assessment result based on the risk value of out-of-range acquisition, the data acquisition security risk value, and the control spoofing risk value.

9. A multi-source heterogeneous data intelligent analysis device, characterized in that, It includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the intelligent analysis method for multi-source heterogeneous data as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, Used to store computer programs, wherein the computer programs, when executed by a processor, implement the intelligent analysis method for multi-source heterogeneous data as described in any one of claims 1-7.

Images