36 results about "Data discovery" patented technology

A multimodal content management system having a block-based data structure can include a query engine configured to perform automatic data discovery for natural language queries. The system can generate and render, at a computing device, a page comprising a graphical user interface (GUI) with a displayable item from a first block of a block-based data structure. The system can generate and bind, to the page, a schema definition comprising a first reference to the first block and a second reference to a set of blocks, wherein the first block is relationally linked to the set of blocks via the second reference. The system can use at least a portion of a natural language prompt, received at the GUI, to generate an input feature for a large language model, the input feature having a schema-question pair that includes at least a portion of the schema definition. The large language model can generate a query configured to operate on the block-based data structure.

The invention provides a sensitive data discovery method based on complex semantic analysis, which innovatively introduces high-dimensional sparse Fourier transform (HSFT), maps semantic characteristics to a frequency domain space, captures semantic change frequency characteristics (such as high-frequency mutation characteristics of sensitive information) which are difficult to recognize by traditional deep learning, and improves the recognition accuracy of the sensitive data. Sensitive information and non-sensitive information in similar contexts are effectively distinguished, and the problems of high misjudgment rate, lack of dynamic adaptability and unreliable risk assessment in complex contexts in the prior art are solved.

Artificial-intelligence computer-implemented processes and machines predict whether personal data may be present in structured software based on metadata field(s) contained therein. Natural language processing preprocesses input strings corresponding to the metadata field(s) into normalized input sequence(s). Individual characters in the sequence(s) are embedded into fixed-dimension vectors of real numbers. Bidirectional LSTM(s) or other machine-learning algorithm(s) are utilized to generate forward and backward contextualization(s). Neural network output(s) are provided based on element-wise averaging or feed forwarding based on the contextualization(s) in order to predict whether one or more value fields corresponding to the metadata field(s) may contain personal data.

A data platform for providing large scale data discovery with near real time data cataloguing and detailed lineage. Data is received from one or more sources, and the received data is written to a data storage systems. The received data is processed at the data storage systems using sensors at the data storage systems. Metadata and data definitions associated with the data are sent to a data service in near real-time. A data catalog is generated at the data service based on the metadata and data definitions associated with the data. Data lineage for the data is created at the data service. The data catalog and data lineage are presented at a data discovery interface.

Techniques for database management are described. A database management system may transmit a request for a data management system of a database to provide a set of metadata attributes for structured data within the database, and may receive a set of metadata attributes for the structured data within the database. The data management system may perform a pattern matching procedure to evaluate the set of metadata attributes for the structured data within the database against one or more patterns associated with a data type to determine one or more locations within the database that include structured data of the data type. Based on the pattern matching procedure, the data management system may output an indication that the one or more locations within the database include structured data of the data type.

The application provides a sensitive data discovery method based on complex semantic analysis, which innovatively introduces high-dimensional sparse Fourier transform (HSFT), maps semantic features to a frequency domain space, captures semantic change frequency characteristics (such as high-frequency mutation characteristics of sensitive information) that are difficult to identify by traditional deep learning, effectively distinguishes sensitive and non-sensitive information in similar contexts, and solves the problems of high misjudgment rate, lack of dynamic adaptability and unreliable risk assessment in the prior art in complex contexts.

The application discloses a tokamak discharge modeling system based on a bidirectional long short-term memory neural network, which is composed of multiple modules, and the modules are low-coupling functional units.The modules at least include a data transfer module, a Batch data access input module, a training self-defined parameter module, a bidirectional long short-term memory neural network modeling module and a data visualization module.The whole system architecture is mainly divided into two paths, namely data training and data modeling.Using the visualization technology, the tokamak discharge experiment personnel can refer to the discharge modeling results and visualize the model training process.The application can one-key type in the experiment proposal stage to model the whole process of the tokamak discharge curve in advance, so that the experiment personnel and the proposal design personnel can check the validity and rationality of the proposal.The application can also be used for assisting data discovery after the experiment.

The application discloses an industrial sensitive data security management platform and method, relates to the field of industrial data security management, and collects multi-dimensional industrial data source information through a data discovery and classification grading module, fuses AI deep learning and traditional matching algorithm to analyze data sensitive information, obtains data sensitivity judgment results and business value evaluation, and constructs a multi-dimensional classification grading matrix and a visual data asset map; an access control strategy is generated through a differentiated protection module; global protection measures are deployed based on the access control strategy, permission control, data encryption storage transmission and differentiated desensitization are implemented; the protection effect is verified through a continuous monitoring and auditing module, abnormal behavior is monitored and traced, and a response and recovery module responds to industrial sensitive data security events and restores business; the problems of difficulty in controlling industrial sensitive data in the whole life cycle and insufficient protection pertinence are effectively solved, and the efficiency and stability of data security protection are improved.

The application discloses a kind of dimension self-adapting human behavior prediction method and equipment, the method first acquires the D-dimensional time series containing human behavior, carries out modal decomposition processing, obtains each dimension mode.Secondly, for each mode, the internal relationship between the implied state, activity contribution and observation event is constructed based on the dynamics equation.Then estimate each parameter of the dynamics equation, and construct total parameter set.Finally, the prediction sequence of each mode is inferred based on the total parameter set, and is reconstructed as D-dimensional time series, realize the prediction of human behavior.In the equipment, the central processing module processes the human behavior data collected by the perception module, the abnormal behavior control interaction module alarms, the communication module is used to send the abnormal data back to the personal computer end, and the data is transmitted to the storage module for saving when power failure or network interruption.The application does not need prior knowledge when carrying out human behavior prediction, low cost and high accuracy.

The application discloses a Web table abnormal data discovery method based on a text semantic mapping relationship. The application aims at discovering abnormal data with fuzzy or even wrong semantic information in a Web table. The method mainly comprises three parts: a semantic representation module, a column type inference module and an error discovery module. First, the semantic representation module represents the meaning of cell text. For a cell in a table, the string text in the cell is represented as a semantic vector according to context information. Then, the column type inference module infers the type of the column where the cell is located, and obtains the mode information of the column. Finally, based on the mapping relationship between the column type and the semantic vector of the cell text of the main column cell and the target cell, abnormal data in the table is discovered and labeled.

A method utilizes data discovery to support visual analysis of a dataset. A user selects a data source, and the method presents a natural language interface for analyzing the data source. The user specifies an incomplete natural language command directed to the data source, and the method correlates words in the incomplete natural language command to data fields in the data source. The method determines a data type of the data fields and a range of data values of the data fields. From the data type and the range of data values, the method presents one or more autocomplete options for the incomplete natural language command. Each option includes respective text and a respective corresponding visual graphic. The user selects one of the autocomplete options, and the method forms a complete natural language command. The method then displays a data visualization in accordance with the complete natural language command.

This invention discloses a highly robust web crawler system, belonging to the fields of big data acquisition and artificial intelligence technology. The system is a closed-loop intelligent system with multiple modules working collaboratively. It includes a multi-data source database construction and pre-training module, a real-time monitoring module for crawler operation status, a data semantic analysis and association mining module, a data request-matching evaluation and parameter optimization module, and a dataset generation and delivery module. Through tight coupling between modules and effective transmission of data flow and control flow, the system achieves full-process automation and intelligence from data discovery, intelligent crawling, dynamic adaptation to high-quality delivery. The system features high robustness and anti-crawling capabilities, strong data semantic understanding and association accuracy, and low manual costs.

A system and method of mixed-reality visualization and analysis of multi-dimensional segmentation for a supply chain network. Embodiments include a supply chain network having supply chain entities and a planner. The planner accesses input data relating to the supply chain entities, discovers features related to the input data, pre-processes the input data and features, performs multi-dimension segmentation on the input data, computes the importance of the features, generates a multi-dimension segmentation visualization, and displays the multi-dimension segmentation visualization. The planner further assigns policy parameters to the multi-dimension segmentation performed on the input data, detects outliers in the multi-dimension segmentation visualization, generates a mixed-reality visualization having clusters and segmentation data displayed on three-dimensional mixed-reality objects. Embodiments further including a mixed-reality visualization system that displays a quantity of the three-dimensional mixed-reality objects equal to the number of dimensions of the segmentation data.

The invention discloses a security authentication method and system based on multi-modal biological feature fusion, and relates to the technical field of biological feature recognition. Environmental factor influence performance index data sets are constructed and grouped by using a clustering algorithm; determining an environmental condition classification set containing groups with insufficient light and normal noise, analyzing a change trend by adopting a sliding window method aiming at performance fluctuation under the conditions of insufficient light, noise interference and the like, generating a core adaptation rule set, and verifying corresponding adaptation adjustment parameters through a simulation environment to obtain a core adaptation rule set; finally, a biological recognition authentication output rule set is formed, a comprehensive analysis framework is constructed, various expression data in the authentication process are systematically monitored and combed, a performance change rule is found through the data, the authentication stability and accuracy in a complex environment are ensured, and the authentication efficiency is improved. And the adaptability and the reliability of a biological recognition technology are remarkably improved.

Method for account-specific discovery by a discoverer device pre-provided with (i) a first encryption seed, (ii) a second encryption seed, (iii) a doubly encrypted metadata encryption key (MEK) and (iv) encrypted metadata relating to an account configured on an advertiser device.The discoverer device (a) detects a broadcast advertising message carrying an encrypted device encryption key (DEK) associated with the advertiser device, (b) uses the first encryption seed to decrypt the encrypted DEK and thereby reveal the DEK, (c) maps the first encryption seed to the second encryption seed, the doubly encrypted MEK, and the encrypted metadata, and (d) accordingly uses (i) the DEK and the second encryption seed cooperatively as a basis for decrypting the doubly encrypted MEK and thus revealing the MEK, and (ii) uses the revealed MEK as a basis for decrypting the encrypted metadata and thus revealing the metadata relating to the account configured on the advertiser device.

Key vault access security migration is provided, including a computing device receiving key vault information. The key vault information is received from at least one entity operating a user computing device via an initialized release pipeline. Further, at least some of the key vault information is processed to determine a first access security model, including permissions to a cryptographic object for access to a respective technical resource. Data discovery determines cryptographic object permission and the computing device determines a role assignment that includes a security principal, at least one of a plurality of permissions, and the respective technical resource. Further, the computing device migrates access to the key vault for the entity from the first access security model to the second access security model. Access to the key is enabled as a function of the second access security model.

Various methods, apparatuses / systems, and media for implementing a data discovery module are disclosed. A repository includes one or more memories that stores application code for each application among a plurality of applications. A processor is operatively connected to the repository via a communication network. The processor scans the application source code for each application among the plurality of applications; identifies, in response to scanning, all technical assets and their relationships within each application; harvests technical metadata from the technical assets and their relationships to identify what information is used, stored, created, and moved by the application; implements machine learning algorithms to automatically assign descriptive and administrative metadata at a field level; loads the assigned descriptive and administrative metadata into an enterprise data catalog; and creates, in response to loading, a knowledge map, thereby providing a fine-grain level understanding of data within the technical assets.

The application discloses a scheme for automatically discovering the similarity between different table data of a relational database based on a Bloom filter algorithm, reads all table information from the relational database, forms a table information dictionary and stores the table information dictionary; a filter is established based on a unique value set of N by using the Bloom filter algorithm; the established Bloom filter is transmitted to a unique value set of M, the value is filtered, and a return result set {R1, R2, R3, … Rn: R epsilon (True, False)} is counted; a probability W is counted: ∑R{true} / (R{true|false}); when W>ɑ (ɑ is a given threshold value), it is determined that M and N are similar, otherwise, M and N are not similar. The application can analyze all tables and all fields, and judge the data similarity between tables without relying on any artificial rules, artificial judgment or predefined instructions, and can be used for main data discovery, data reference relationship generation and data table relationship construction in a data management process, so that the traditional data management process is changed from manual to intelligent, and the management accuracy and management efficiency are effectively improved.

This disclosure presents a method and system for automatically discovering sensitive data, comprising: monitoring a task queue to obtain sensitive data discovery tasks to be processed; parsing the sensitive data discovery tasks, dynamically adapting connection strategies using a database fingerprinting mechanism, and connecting to a target database; extracting configuration information, determining the target database table name and target field name, generating and executing an intelligent sampling strategy and data query SQL statement, scanning and obtaining data samples, performing sensitive information pattern matching and identification, and generating discovery results and a data evolution tracking system; writing the discovery results into a result table and generating a data lineage graph; updating the execution progress of the sensitive data discovery tasks, closing the database connection after execution, and updating the status of the sensitive data discovery tasks to complete. This method can improve the efficiency, accuracy, and traceability of sensitive data discovery.

The application relates to a method for discovering associated nodes of a springboard node. By analyzing the flow data of the springboard node in a botnet, nodes with highly similar behaviors are discovered from the nodes connected to the springboard node, thereby discovering multiple associated nodes belonging to the same botnet, positioning the C&C server node possibly at the upper level, and providing help for subsequent trace analysis, discovery and prevention of botnet threats. The application can extract important features representing the behaviors of network nodes through analysis of network flow data, input the features into a well-constructed program after pretreatment, complete the discovery of nodes with similar behaviors, and output the results. By drawing a flow curve of the communication between highly suspicious nodes and the springboard node and visually displaying the flow curve, the upper and lower control relationship of the suspicious IP pair is further verified, and the discovery of associated nodes of the same botnet and attack prevention are realized.

A system and method of autonomous multi-dimensional segmentation for a supply chain network. Embodiments include a supply chain network of one or more supply chain entities, a segmentation planner having a computer and memory, the segmentation planner configured to access input data relating to one or more supply chain entities, discover one or more features related to the input data, pre-process the input data and features, perform multi-dimension segmentation on the input data, compute the importance of the one or more features, generate a multi-dimension segmentation visualization, assign policy parameters to the multi-dimension segmentation performed on the input data.

The system according to the embodiment aims to detect a corporate crisis early and respond to it quickly and effectively. [Solution] A system according to an embodiment includes a collection unit, an analysis unit, a discovery unit, an alert unit, a proposal unit, and a generation unit. The collection unit collects data. The analysis unit analyzes the data collected by the collection unit. The discovery unit detects crises early based on the data analyzed by the analysis unit. The alert unit issues an alert for a crisis detected by the discovery unit. The proposal unit proposes countermeasures based on the alert issued by the alert unit. The generation unit generates a draft for effective communication based on the countermeasures proposed by the proposal unit.

This disclosure describes one or more implementations of systems, non-transitory computer-readable media, and methods that utilize a repository of metadata-based recommendations to classify data sources using metadata from the data sources. For example, the disclosed systems can generate a repository of metadata-based recommendations that indicate recommended classifications for objects within data sources through metadata associated with a data source schema. In some instances, the disclosed systems identify metadata from a data source schema associated with the data source. Subsequently, the disclosed systems can match the identified metadata to a metadata-based recommendation via metadata mappings in the metadata-based recommendation repository to select a metadata-based recommendation. Furthermore, the disclosed systems can also utilize a classifier model to generate predicted labels for the data source and update the metadata-based recommendation repository with a mapping between the predicted labels and metadata corresponding to the data source schema of the data source.

The system according to this embodiment aims to utilize data from all assets to discover cross-sectional insights and propose solutions. [Solution] The system according to the embodiment comprises a collection unit, an analysis unit, a discovery unit, and a proposal unit. The collection unit collects customer understanding data, behavioral logs, and statistical data. The analysis unit analyzes the data collected by the collection unit. The discovery unit discovers insights based on the data analyzed by the analysis unit. The proposal unit makes solution proposals based on the insights discovered by the discovery unit.

The application discloses a context scene data recognition method and device, and the method comprises the steps: inputting the preceding data and the following data in the context text data into a single sentence topic model respectively to obtain preceding topic representation and following topic representation; inputting the context text data into a context topic model to obtain context topic representation; calculating the similarity of any two of the preceding topic representation, the following topic representation and the context topic representation; judging whether the similarity meets a threshold condition, and if yes, judging the preceding data and the following data as context scene data. The context scene data recognition method improves the sufficiency of context scene data discovery through two rounds of context data discovery process, and reduces the manpower, time and cost consumed in the context scene data labeling process.

The application discloses a medical research data space-oriented data discovery and privacy computing cooperation method, device, medium and product; the method comprises the following steps: establishing a contract unique identifier for medical research data; opening data structure information of medical research data not containing real data entities to a data user; generating sample data corresponding to the medical research data, and providing the sample data to the data user after opening the data structure information; receiving a probe condition input by the data user on the sample data, and associating the probe condition to the contract unique identifier; converting the probe condition associated with the contract unique identifier into a data delivery range; after the contract takes effect, performing a real query operation on original real data corresponding to the medical research data according to the contract unique identifier and the data delivery range, obtaining real data results, and connecting the real data results to a data space privacy computing platform; and the safety of the medical research data is ensured.

The present application relates to the technical field of financial data security, and discloses an artificial intelligence-based financial data security analysis system, which comprises a financial data collection unit, an artificial intelligence detection unit, a person portrait analysis unit, an abnormal threshold management unit and a data security management unit. The financial data collection unit is used for collecting financial data from a financial data source and distributing the financial data according to the users. The artificial intelligence detection unit is used for extracting features from the financial data, obtaining normal features and malicious features, and establishing an artificial intelligence detection end according to the normal features and the malicious features. The artificial intelligence detection end is constructed in multiple steps and optimized by using multiple model training, and has reliable performance. The data monitoring module compares real-time data with malicious feature data in real time with the help of the detection end, takes safety protection measures such as stopping transactions and freezing accounts when malicious features are found, prevents risks in time, and protects the safety of user funds and financial institutions.

The application relates to a network space knowledge extraction method and device based on rule-enhanced prompt learning, which comprises the following steps: acquiring text data to be extracted in a network space security field and a prompt input; splitting the prompt input into a subject prompt, a relationship prompt and an object prompt; connecting sub-prompts of a conditional function related to an ontology rule by using a logical rule with an and normal form to obtain a task-specific prompt for the text to be extracted; performing network space knowledge extraction on the text data to be extracted by using the task-specific prompt and a pre-trained language model trained to output entities and relationships of a current task in the network space security field; and using the entities and relationships to determine threat intelligence data in the current network space security field. The application realizes efficient extraction of entities and relationships in the network space security field by using a prompt tuning technology, and improves the efficiency of threat intelligence data discovery and acquisition in the network space security field.

A system and method for data discovery. A method includes performing a scan of a plurality of snapshots, each snapshot corresponding to a respective disk of a plurality of disks; identifying a plurality of data store files in the plurality of disks based on file metadata found during the scan; and detecting at least one data store based on the identified plurality of data store files, wherein each of the at least one data store is in a disk of the plurality of disks including one of the plurality of data store files.

The application discloses a kind of oil and gas field data change notification method, device and system, the method includes: when the oil and gas field data change event occurs, when event change type is the type that needs to be notified, generate oil and gas field event data body, and generate abstract information;According to oil and gas field event data body, abstract information and the identity token of oil and gas field data center, generate data change request, and send to oil and gas field data service control system, so that oil and gas field data service control system is verified to abstract information and the identity token of oil and gas field data center, after passing, according to data exchange model type, the changed oil and gas field data body in data change request is checked, and in check through, the oil and gas field data that meets data discovery rule is sent to corresponding oil and gas work area.The application improves the performance of oil and gas field data center and oil and gas field data service control system in the case where the message notification efficiency is guaranteed.