178 results about "Independent data" patented technology

A heterogeneous storage-oriented hot and cold data management method and system, wherein hot and cold data are managed by means of the following steps: at table granularity, storing data from each table into an independent data bucket in the form of object storage, each data bucket comprising a plurality of partitions, the data being randomly distributed in each partition, and the consistency of replicas of the partitioned data being ensured by means of a distributed consensus algorithm; collecting statistics about the access frequencies of the partitions in one data bucket; when the access frequency of one partition reaches a set threshold, adding a new replica to a target storage medium by means of a membership change in the distributed consensus algorithm; and when data synchronization between the new replica and an old replica is completed, detecting the old replica. In this way, the migration of the partitioned data on different storage media is realized, and data cooling and heating are implemented. The method and system can maximize the use efficiency of high-performance storage media, ensure the access efficiency of users, and can reduce the costs of data migration and maintenance.

The invention discloses a multi-mode fusion data processing method and device, a medium, electronic equipment and a program product, belongs to the technical field of computers, and can reduce the data processing complexity and meet the service requirement of high timeliness when responding to application service to integrate multi-mode data. The multi-mode fusion data processing method comprises the following steps: receiving a multi-mode data retrieval statement, analyzing the multi-mode data retrieval statement, and translating the multi-mode data retrieval statement into single-mode data retrieval statements corresponding to a plurality of modes respectively; according to each single-mode data retrieval statement, performing data retrieval in a corresponding single-mode database in the multi-mode database to obtain target data of different modalities; wherein single-mode databases corresponding to a plurality of modes are maintained in the multi-mode database, and independent data management is carried out on each single-mode database; and performing feature fusion and modal alignment processing on the retrieved target data of different modalities to generate a unified multi-modal data representation, and feeding back the unified multi-modal data representation as a retrieval result.

The invention provides a Hadoop cluster incremental backup recovery method based on data block optimization, and relates to the technical field of data management, and the method comprises the following steps: obtaining distribution state information and historical change records of data blocks in a distributed file system, extracting space-time correlation characteristics, and constructing a data block dependency graph containing dependency direction and dependency intensity; identifying the changed data blocks, performing topological sorting, and identifying key path data blocks and independent data block clusters; calculating a distribution entropy value according to a copy distribution position, and determining a backup granularity type; a backup execution sequence is generated according to a topological sorting result, key path data blocks are backed up preferentially, independent data block clusters are backed up in parallel, and backup metadata containing dependency relationship identifiers and distribution entropy values are generated; and performing dependency consistency verification after recovery based on the dependency relationship identifier. According to the method, the backup recovery efficiency is improved, the data consistency is guaranteed, and the storage overhead is reduced.

The invention relates to a micro-service architecture system based on business function and data double isolation, and the system comprises a business function splitting module which splits a single application into independent micro-services; the data physical isolation module is used for distributing an independent database instance for each micro-service; the dynamic routing control module is used for dynamically distributing flow through a load sensing strategy and integrating a fusing mechanism; and the gray release module supports flow hierarchical diversion and abnormal automatic rollback, and ensures safe switching of service versions. According to the invention, by realizing double isolation of service functions and data, the independence and the expandability of the system are improved, and the data coupling degree and the maintenance cost are reduced; the stability and reliability of the system are ensured, and the risk in the migration process is reduced; according to the method, rapid iteration and continuous delivery can be realized, the development and deployment period of new functions is accelerated, and the user experience is improved through efficient service governance and data consistency guarantee, so that the system can better cope with challenges of high concurrent access and distributed transaction processing.

The invention provides a data migration method, which is applied to a data migration processing device and comprises the following steps: polling each cache region of a doorbell cache, determining a target accelerator corresponding to an accelerator identifier under the condition that the accelerator identifier is acquired from a target cache region, and acquiring a first target address of first target data from the target cache region; the first target data is to-be-moved data, and the accelerator identifier and the first target address are sent by the host; and sending a first data migration instruction carrying the first target address and the address of the target accelerator to the DMA, so that the DMA migrates the first target data from the host to the target accelerator. The independent data migration processing device is used for controlling DMA to achieve cache space management, participation of a CPU can be reduced, CPU resource consumption caused by cache management is reduced, the cache management process and the CPU are further decoupled, and the flexibility of accelerator cache space management is improved. The invention further provides a data migration processing device, a chip and electronic equipment.

The invention discloses a neural patient rehabilitation training method based on task training and brain-computer interaction, and belongs to the technical field of rehabilitation training. The method comprises the following steps: acquiring data signals of a patient in current rehabilitation training in real time, performing segmentation processing to obtain a plurality of independent data blocks, performing frequency domain conversion on the independent data blocks, extracting time-frequency features, and calculating KL divergence between adjacent data blocks; if the KL divergence of any adjacent independent data block is larger than or equal to a preset threshold value, historical data signals are obtained, the motion decoding model is corrected, motion decoding is carried out on the patient according to the corrected motion decoding model, rehabilitation training is carried out on the patient according to the motion decoding model, and rehabilitation training is carried out on the patient. According to the method, the corresponding model is selected according to the current rehabilitation training task, dynamic adjustment is performed in combination with real-time signal features, high matching of the decoding result and the training target is ensured, and the rehabilitation training effect of the patient is improved.

The invention discloses a multi-terminal user data security grading protection system, which comprises three independent data storage domains, namely a user terminal, a platform terminal and a hospital terminal, the user side is configured to store personalized data and interaction records; the platform end is configured to encrypt and store operation data and non-sensitive data in a partition manner by using a user identity identifier; the hospital end is configured to uniquely store sensitive diagnosis and treatment data and prohibit transmission of a complete data copy to an external domain; the user side obtains the authentication token issued by the platform side through two-factor authentication; when the user side accesses the data of the hospital side, the hospital side sends an authentication token to the platform side to verify the identity legality, and if the verification is passed, a desensitization data index is returned; and generating a dynamic access link based on the index, so that a user can check the non-downloadable data rendered by the hospital end page service within a limited time through the access link. According to the method, the privacy leakage risk is reduced, redundant cache is effectively eliminated, and the cost of multi-place storage is reduced.

The invention discloses an API publishing method based on a security data sandbox, and relates to the technical field of computer networks. Comprising the steps that 1, API online development is carried out, 2, a security data sandbox is deployed and used for data security isolation of the API, an independent data sandbox environment is created, each sandbox has a unique identifier and access permission control, a user is allowed to carry out data preprocessing and model training in the isolated sandbox environment according to the unique identifier and the access permission control, and the data security isolation is carried out. And step 3, using the APISIX as an API gateway to carry out proxy release and management on the API: using the API gateway to carry out dynamic routing and load balancing management, carrying out gray release to allow part of flow to gradually migrate to a new version API, supporting a blue-green deployment mode, ensuring that service upgrade is not interrupted, formulating a multi-dimensional security authentication mode and access control of the API, and carrying out proxy release and management on the API. And meanwhile, DDoS protection and rate limitation are carried out, and log auditing and monitoring alarm are carried out.

The invention discloses a three-layer micro-service architecture design and deployment method, which comprises the following steps of: constructing a service registration layer, and deploying a registration center to obtain a service registration information set; service discovery, routing distribution, load balancing and fusing control are executed based on the service registration information set, and a service governance configuration set is constructed; deploying a decoupled function service module, and configuring an independent database and cache to form a micro-service operation system structure; establishing a configuration management mechanism, and generating a configuration change request information set; based on the configuration change request information set, training a batch ordered lifting CatBoost model, and outputting configuration risk level information; obtaining hybrid deployment result information based on the micro-service operation system structure and the configuration risk level information; and constructing a full-link tracking mechanism based on the service governance configuration set and the service deployment topological structure to obtain an operation observation information set. According to the invention, a three-layer architecture and risk prediction are adopted, and stable and intelligent deployment of micro-services is realized.

A methods, systems, and devices for secure linking and matching of data elements across independent data systems of a first and a second enterprise are disclosed. The system includes a first private data system having a first private database that may include all attributes known to the first enterprise A reference data system having a reference database and is also part of the first enterprise. The reference database may contain a plurality of identity data records that only contain identity attributes that are used for resolving and verifying identities of unknown data records that are external to the reference data system. A first set of query attributes may be used determine whether a subject represented by the first set of query attributes associates uniquely with a subject of any private data record stored in the first private database. A similar comparison may be made in the reference database.

The invention relates to the technical field of data retrieval, in particular to a data retrieval method based on multivariate matrix fusion, which comprises the following steps: S1, multivariate classification: dividing a whole group of source data into a plurality of independent data elements according to a unified classification standard by defining a classification dimension, and controlling the classification granularity, namely the classification thickness in the classification process, the granularity is ensured to be reasonable through consistency verification, and a two-dimensional matrix construction basis is established for subsequent elements; s2, constructing a matrix, and obtaining independent data elements based on the multivariate classification in S1; s3, similar interval judgment based on the multi-type matrix constructed in S2; according to the scheme, data elements are split according to a unified standard, a multi-type matrix is constructed, instruction analysis and similar interval judgment are standardized, similarity levels are calculated in a weighted fusion mode, and an optimization result is presented, so that structured processing of multi-dimensional data, precise collaborative retrieval and decision-making auxiliary support are achieved; the problems that a retrieval method lacks systematic classification in data processing and is insufficient in multi-dimensional collaboration are solved.

The invention relates to the technical field of system database management, in particular to an intelligent monitoring analysis method and system for gate opening and closing working condition data, and the method comprises the steps: obtaining fault records from a historical fault data storage library of a specified gate, and constructing a fault feature library; acquiring historical monitoring data of the sensor group, performing pattern recognition and statistical analysis to form a fault feature weight table, and storing the fault feature weight table in a data cleaning database; and classifying the sensor group through a database index technology by utilizing the fault feature weight table, the external prediction data and the sensor installation position data, and storing a classification result into a sensor level index table. And a distributed data intelligent monitoring system is formed through N independent data processing units in combination with a database interface protocol. The data processing unit comprises a data acquisition layer, a data real-time analysis layer, a data cleaning and quality control layer and a data storage and retrieval layer, and high-efficiency processing and retrieval of data are ensured.

The invention relates to the technical field of voice processing, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a voice-driven information recording and checking method, device, equipment and medium. Key fields are extracted by applying a semantic analysis strategy to form preliminary structured data, an independent data system is connected to verify authenticity and generate data conflict information, and automatic error correction is executed based on the conflict information to generate updated structured data; and according to the user role matching output template, filling the template with the updated structured data to generate display content, and pushing the display content to target terminal equipment. Structured data generation is realized through voice input and semantic analysis, the data accuracy is improved by combining cross-system verification and automatic error correction, personalized display and real-time pushing are realized by utilizing a role matching template, and a closed loop of acquisition, verification and display is formed, so that the efficiency of information recording and management is improved.

The embodiment of the invention provides a data collection method and system, electronic equipment and a storage medium, and is applied to the technical field of industrial communication. According to the data acquisition method and system, the electronic equipment and the storage medium provided by the embodiment of the invention, the register objects with the same acquisition period, the same slave station ID, the same function code and continuous register addresses are aggregated in the same second register aggregation group, and the data request frame is generated for the same second register aggregation group; the register objects in the second register aggregation group can share one data request frame. Compared with the prior art in which each register object generates an independent data request frame, the data acquisition method and device have the advantages that the number of the data request frames can be obviously reduced, so that excessive system resources are prevented from being occupied, and the data acquisition efficiency is further improved.

The invention provides an Internet of Things event center distributed alarm processing method based on Redis Stream, belongs to the field of Internet of Things (IoT) data processing, and realizes millisecond-level event processing by utilizing the high-performance streaming consumption capability of the Redis Stream and combining a distributed rule matching engine and a cross-device data dynamic association mechanism. Comprising the following steps that: Internet of Things terminal equipment writes data into Redis Stream according to a stream: Model: DeviceId mode to form an independent data stream; an event consumer reads the data from the stream, performs matching based on a preloaded rule, and supports cross-device linkage calculation through a metadata relation table and an SQL template; real-time duplicate removal and full-life-cycle management are carried out on alarm events through a dual convergence mechanism based on a database and a cache, and repeated alarms are eliminated; and finally, through multi-channel push and push state write-back, a closed-loop alarm system is constructed. The system adopts a layered architecture, comprises an event acquisition layer, a rule processing layer, an event management layer, a push service layer and an asynchronous processing layer, has the characteristics of high throughput, strong orderliness and high reliability, and can effectively improve the alarm processing efficiency and accuracy in a large-scale Internet of Things scene.

The invention discloses a geological data security sharing method and system based on a multi-tenant architecture, and relates to the technical field of geological data management.The sharing system comprises a physical layering architecture, a logic isolation mechanism, a hierarchical management system and a data sharing mechanism, according to the geological data security sharing method and system based on the multi-tenant architecture, through the system and method, by constructing a standardized data interface, an authority grading mechanism and a cross-library collaboration platform, the sharing efficiency and the application value of geological data are remarkably improved, and under a sub-library architecture, the sharing efficiency and the application value of the geological data are greatly improved. The geological data of different units are stored through physical isolation, meanwhile, cross-database data sharing is achieved through unified data dictionary and metadata management, independent data access spaces are provided for different units in a multi-tenant mode, and tenants are allowed to share specific data sets within an authorization range.

The invention relates to the technical field of marine ecology and water color remote sensing, in particular to a global chlorophyll concentration three-dimensional data reconstruction method and system and a computer program. The method comprises the following steps: acquiring multi-source data; carrying out space-time matching on multi-source and multi-dimensional data; processing and standardizing multi-type data; filtering and preprocessing abnormal data; constructing an inversion model; training a chlorophyll concentration inversion model; checking an independent data set model; making a spatio-temporal data set to be inverted; according to the method, global chlorophyll concentration three-dimensional data reconstruction capacity is remarkably improved, three-dimensional daily data production of global chlorophyll concentration data is supported, and high-reliability data support is provided for marine ecological research and estimation of global three-dimensional chlorophyll concentration data.

The invention relates to the technical field of carbon emission, and discloses a multi-source data fused personal carbon footprint real-time visual feedback system, which comprises a data acquisition module used for acquiring multi-source heterogeneous data from a plurality of independent data sources and representing user behaviors in real time; the processing module is connected with the data acquisition module and is used for carrying out real-time estimation on a composite state vector based on a preset state space model according to the multi-source heterogeneous data, and the composite state vector at least comprises a current activity state of a user and personalized carbon emission factors representing emission characteristics of the user; and calculating the personal carbon footprint based on the estimation result of the composite state vector. According to the method, the unified dynamic state space model is constructed, and the activity state, the physical parameters and the personalized emission factors of the user are integrated into the composite state vector for joint real-time estimation, so that the real-time performance, the continuity and the accuracy of personal carbon footprint accounting are remarkably improved.

The invention discloses a hospital examination and inspection result sharing method based on big data, and relates to the technical field of hospital examination and inspection result sharing. Examination data and inspection data are respectively stored in independent databases, and the two types of data are intelligently linked by using a unique identity label of a patient and a time stamp; the format, the time sequence and the semantic information are accurately matched to form a continuous and visual comprehensive disease course record; introducing a deep learning model, carrying out natural language processing on an unstructured text in the inspection data, and automatically converting an image description and a diagnosis report into structured data and a predefined semantic tag; according to the method, the workload of manual structuring is reduced, distributed storage, dynamic updating and multi-dimensional index query modules are constructed based on a big data analysis platform, real-time retrieval and joint query are conducted on comprehensive disease course data records, meanwhile, an early warning report is generated through an anomaly detection algorithm, and a basis is provided for clinical decision making.

The invention discloses an industrial chain logic deduction method, computer equipment and a program product, and the method comprises the steps: constructing a data asset library which comprises charts, data indexes and research reports as independent data assets; the method comprises the following steps: constructing a visual editing framework based on a Web technology, wherein the framework provides a predefined graphic element library and canvas; in the canvas, dragging elements in the graphic element library and establishing connection to construct an industrial chain logic deduction graph; carrying out association binding on nodes in the logic deduction graph and charts, data indexes or research reports in a data asset library; and performing logic deduction and demonstration in response to user operation based on the bound logic deduction graph. By supporting interactive demonstration and path multiplexing based on the logic deduction graph, effective transmission of analysis thoughts and continuous precipitation of knowledge are realized, and the depth and efficiency of industrial chain research are comprehensively improved.

The application discloses an antiviral peptide prediction method and system based on federated learning and a neural network, utilizes multiple independent data sets to respectively perform training on a prediction model, constructs a double-channel deep neural network through the model, extracts features of different dimensions from original variable-length sequence data, analyzes peptide segment sequences from two aspects of sequence data and evolution levels, and finally transmits model gradients and parameters obtained to an intermediate server through encryption technology; the server aggregates the transmitted model parameters to form an intermediate model without contacting the data itself; the server transmits parameters of the intermediate model to participants, completes updating of participant models, and the above steps are circularly performed until model training is completed. The combination of federated learning and the neural network can accurately and quickly predict the antiviral activity of peptide segments under the premise of guaranteeing data security.

Systems, methods, and other embodiments associated with a centralized product catalog with standardization from independent data sources are described. In one embodiment, a computing system is configured to access and download product information for a plurality of products that is maintained in a non-standard format. Metadata is identified and extracted. Feature vectors may be generated from the metadata and used to predict and assign class codes to products. The product attributes are extracted from each product, which include a non-standard product description. The non-standard product description is converted into a standardized text description. Standardized records are generated for each product record and a centralized database is generated with the standardized records.

The invention discloses a dangerous goods vehicle complex environment monitoring method and system based on satellite positioning, and relates to the technical field of vehicle monitoring, and the method comprises the steps: obtaining the historical monitoring data of a dangerous goods vehicle, and carrying out the information extraction to obtain historical satellite positioning data, historical dangerous goods vehicle motion data and historical geographical environment data; constructing a combined positioning module according to the historical satellite positioning data and the historical dangerous goods vehicle motion data; according to the combined positioning module and the historical geographical environment data, constructing a risk assessment early warning model; and obtaining current satellite positioning data, current dangerous goods vehicle motion data and current geographical environment data, and inputting the data into the risk assessment early warning model to obtain a risk assessment early warning result to perform complex environment monitoring on the dangerous goods vehicle. The problems that a traditional monitoring method is insufficient in positioning continuity, single in risk assessment and mutually independent in data information are solved, and multi-dimensional and intelligent risk assessment early warning of the dangerous goods vehicle in a complex environment is achieved.

The invention relates to a quality optimization method for offshore multi-source space-time monitoring data. The method comprises the following steps: acquiring an offshore multi-source space-time monitoring data set; constructing a multi-level evaluation index system; determining the weight of each index in the evaluation index system; obtaining a quality score of each independent data sample; multi-source data collaborative quality evaluation is carried out through a two-stage fusion strategy; and screening the original monitoring data according to the comprehensive quality evaluation, constructing an optimal data set oriented to a specific application scene, inputting the optimal data set into the application model, and outputting an environmental element inversion result. The method has the advantages that progressive quality optimization from samples to platforms and then to application results is achieved in a unified framework, low-quality samples can be effectively removed, and the utilization efficiency of multi-source data and the accuracy of inversion results in offshore ecological environment monitoring can be improved.

The invention discloses a medical Internet of Things data system and method based on digital twinning, and relates to the technical field. Comprising a digital twinborn model building module, a multi-modal characteristic data construction module, an equipment use interval period analysis module, a duration estimation model analysis module, an equipment response demand analysis module and a real-time grading early warning module. The digital twinborn model building module is used for obtaining medical Internet of Things data by utilizing edge calculation based on an Internet of Things protocol to build a medical digital twinborn model; the multi-modal feature data construction module is used for extracting user state data, equipment resource data and environment parameter data recorded in the digital twin model to construct multi-modal feature data taking user information as a main body; the equipment use interval period analysis module is used for analyzing equipment use interval periods of all users corresponding to analysis event records; and the duration estimation model analysis module is used for constructing an equipment standby duration estimation model corresponding to each independent data element.

The invention discloses a distributed file system oriented to consumer-level equipment and a data security management method of the distributed file system, and belongs to the technical field of computer systems. The system comprises a virtual namespace management module used for maintaining a virtual namespace tree formed by combining a local main namespace tree and a remote snapshot sub-tree; the access control and encryption module is used for executing access control based on the attribute set of the virtual namespace tree nodes; and the data processing pipeline module is used for partitioning the file and realizing pipeline operation of transmission and decryption through independent data flow and control flow channels. According to the invention, the access control and the encryption strategy are tightly coupled based on the unified attribute set, so that file re-encryption is not needed when the authority is changed, and the system flexibility is improved; and the security operation delay is hidden through data partitioning and pipeline processing, so that the file access efficiency is improved, and the problem of balance among security, flexibility and high efficiency of the distributed file system of the consumption equipment is effectively solved.

The application discloses a water gate structure stress and seepage flow intelligent monitoring system, and belongs to the technical field of water conservancy projects.The system comprises a stress monitoring module, a seepage flow collecting module and a data processing and analysis module.The stress monitoring module is used for monitoring the stress change of the water gate, including the stress change of key parts, wherein the key parts include a gate pier, a gate and a breast wall.The seepage flow collecting module is used for collecting the seepage pressure and seepage flow of the water gate.The data processing and analysis module is used for processing the stress change, seepage pressure and seepage flow, including cleaning and correction.The application can monitor the stress change of the key parts of the water gate, collect the seepage pressure and seepage flow, obtain more comprehensive and accurate monitoring data, and perform fusion processing on the data, so that the limitation of relatively independent data in traditional monitoring is broken, mutual verification and supplement among the data are realized, the safety state of the water gate is comprehensively evaluated from multiple dimensions, and the accuracy and reliability of monitoring are improved.

The invention discloses a data customization production delivery method and system based on a block chain and cloud, and relates to the technical field of data processing and information security, and the method comprises the steps that a data customization party and a production party register and bind a public chain address in the system; deploying an intelligent contract with an automatic fund transfer function on a public chain based on the transaction demand; the customization party prepares a production environment, including creating a production cloud container and a cloud database, and generating an asymmetric key pair; the producer completes data production in the isolated cloud container, encrypts the whole-process data by using the public key of the customizing party and stores the encrypted data into the private block chain; and the system then starts an independent data verification container, decrypts the data by using a built-in private key, carries out integrity comparison and quality verification, and automatically triggers an intelligent contract to carry out settlement according to a verification result. The problems that the data is easy to reuse, the ownership is unclear and the settlement efficiency is low in data customization production are effectively solved, and safe, credible and efficient data production and delivery are realized.

The invention discloses a unified label index library construction method based on a multi-dimensional application field, and relates to the technical field of financial data management. The method comprises the following steps: extracting a general rule, establishing a reusable label index template, and intensively constructing three types of basic libraries; integrating real-time data of the message center of the head office and local data sources of the branches, and synchronizing customer transaction data by everyday increment ETL; a unified real-time query interface is provided, the real-time query interface is packaged through an Flink + ES engine, and all target data are returned through a single request; setting an independent data item space, a control data authority, sensitive information desensitization processing and a parameterized flexible configuration interface for each branch; and establishing a causal chain between metadata and a future trend thermodynamic diagram based on the business application timeliness iteration library content. According to the unified label index library construction method based on the multi-dimensional application field, the problems of fragmentation, low timeliness and high security risk of branch data assets are solved, and'sharing reuse and business technology fusion 'of data are realized.

The invention discloses an unauthorized business asset discovery method and system based on a multi-source log. The method comprises the following steps: determining a first network entity based on heterogeneous security logs of different data sources; when it is determined that the first network entity does not exist in the configuration management database CMDB, determining that the first network entity is a suspicious network entity; and when it is determined that the suspicious network entity appears in at least two independent data sources, the activity in a preset time window meets a preset persistence condition, and a service port exists, determining that the suspicious network entity is a persistent unauthorized service asset. Therefore, through multiple independent data source conditions, activity persistence conditions and service port conditions, suspicious target entities related to transient temporary connection are excluded, and persistent unauthorized business assets are accurately identified.