94 results about "Data scrubbing" patented technology

The invention provides a cache data cleaning method and device, electronic equipment and a storage medium, and relates to the technical field of computer data storage.The method comprises the steps that a transition controller is used for replacing a fault controller for mirror image recombination, and a temporary mirror image pair is formed; it can be ensured that the basic data redundancy and the read-write function of the storage system are not seriously influenced; when the fault controller is recovered, the complex fault condition can be sensed in real time; once the concurrent fault occurs, the data cleaning process is adjusted according to execution state information of the data cleaning process, team information of the temporary mirror image pair and concurrent fault information of a concurrent fault controller; the data of the transition controller can be effectively cleaned under the condition of concurrent faults, and the fault-tolerant capability of the storage system is improved; when the cache data of the transition controller is cleaned, the overall performance and efficiency of the storage system are improved through an effective fault processing mechanism and accurate data cleaning process adjustment.

The invention provides a three-dimensional visualization expression method for a multi-scale geological structure model, including: collecting geological exploration data which includes geophysical information, drilling information, outcrop survey information, seismic information and geological age data; performing interpolation simulation on the geological exploration data to obtain an exploration sample set; preprocessing the exploration sample set to obtain preprocessed data, the preprocessing process including data cleaning, format unification and noise removal; constructing a multi-scale geological structure model according to the preprocessed data based on a three-dimensional modeling technology; performing scale decomposition on the geological structure model using a multi-scale analysis method to form model representations at different scale levels; and converting the model representations into 3D visualization images using a 3D graphics processing technology. The invention can effectively improve the expression ability of geological model and user experience, and has important practical application value for geological exploration and resource assessment.

The invention discloses a multi-source data fusion and intelligent scheduling method under a virtual power plant platform, relates to the technical field of virtual power plant optimal scheduling, and solves the technical problem that accurate adjustment of energy output is difficult to realize due to lack of a scientific energy scheduling priority evaluation system. According to the method, comprehensive data cleaning and deep fusion are carried out on the multi-source data, the data quality is improved, the potential correlation between the data is mined, the LSTM model and the fusion feature data are combined to establish the short-term load prediction model, the load change trend can be better captured, and the load prediction accuracy is improved. An energy scheduling priority evaluation system is constructed from three dimensions of policy scoring, economical efficiency scoring and adjustment capability scoring, comprehensive scoring and sorting are calculated through weighting, utilization of high-priority energy is guaranteed preferentially, and a refined scheduling strategy is formulated based on priority sorting and a load difference value when a load is short of supply or is over supply.

The invention relates to a system and method for low-voltage judgment of a power failure area based on mobile optical modem equipment, and the method comprises the steps: collecting the static data and offline event data of each mobile optical modem in real time, and storing the data after the data cleaning operation; based on historical offline event data of each mobile optical modem, an initial weight is distributed for each optical modem offline event, and then the offline weight of each mobile optical modem is dynamically adjusted according to the spatial and temporal distribution characteristics of the real-time offline event data of each mobile optical modem; in combination with the offline weight of each mobile optical modem, determining the power failure judgment weight of the corresponding area through space-time scanning statistics; and comparing the power failure judgment weight of each area with a corresponding power failure threshold value to judge whether a power failure fault occurs in the area, and if the power failure fault occurs, outputting position information and power failure warning information corresponding to the area. Compared with the prior art, the method can timely and accurately identify the regional power failure event and determine the fault region.

The invention relates to a method, device and equipment for estimating the position of an elevator, a storage medium and a program product. The method comprises the steps that electric signal collection is conducted on the elevator traction machine, and voltage and current in a target time period are obtained; respectively determining the voltage frequency and the current frequency of the elevator traction machine according to the voltage and the current; the angular velocity of the elevator traction machine is calculated according to the voltage frequency, the current frequency and the number of pole pairs of the motor, and the comprehensive angular velocity is determined based on the voltage effective value corresponding to the voltage and the angular velocity; the start-stop time point of the elevator traction machine is determined, data cleaning is conducted on the comprehensive angular speed according to the start-stop time point, and the target angular speed is obtained; the target angular speed is input into an elevator system mathematical model for displacement calculation, and displacement information of the elevator car is obtained; and the position information of the elevator car is determined based on the displacement information and accumulated displacement information of the elevator car in the ascending or descending stage. By adopting the method, the estimation accuracy of the elevator position can be effectively improved.

The present disclosure provides a method, apparatus and system for monitoring ultra-high frequency partial discharge of a hydro-generator, and belongs to the technical field of hydro-generator partial discharge monitoring. The method includes: cleaning a partial discharge pulse sequence using a cleaning threshold to obtain a valid pulse sequence; performing redundant data filtering on each data unit divided from the valid pulse sequence to obtain a first target pulse sequence for short-period partial discharge monitoring; determining sub-sequences that are partial discharge events from the valid pulse sequence, forming a second target pulse sequence after associating an amplitude statistical feature of the sub-sequences, and storing the second target pulse sequence. The aforementioned method combines data cleaning, redundant data filtering, and partial discharge event identification, which enhances the real-time performance of partial discharge monitoring and records the long-period partial discharge change trend. The corresponding system adopts a multi-buffer zone and multi-processor architecture, thus further improving the real-time performance of partial discharge monitoring.

The embodiment of the invention provides a video memory control method, device, equipment and system and a computer storage medium. The method comprises the steps that in response to an obtained drive upgrading request of an image processing unit (GPU), a before-upgrading page table corresponding to the GPU is obtained, and the before-upgrading page table comprises a mapping relation between a physical address and a virtual address of a GPU video memory; determining a data cleaning queue corresponding to the GPU, wherein the data cleaning queue is used for performing global cleaning on data in a GPU video memory; under the condition that the data cleaning queue is locked, driving upgrading operation is carried out on the GPU based on the driving upgrading request, an upgraded page table corresponding to the GPU is obtained, and the upgraded page table comprises a mapping relation between a physical address and a virtual address of a GPU video memory; and regulating and controlling data in the GPU video memory based on the pre-upgrading page table and the post-upgrading page table. In the embodiment of the invention, the driver upgrading operation of the GPU can be realized without operations such as video memory backup, shutdown, restart and the like.

According to one embodiment of the present invention, a computer-implemented method for dynamically altering a frequency at which data scrubbing is performed on a memory device is disclosed. The computer-implemented method includes monitoring at least one of a temperature and a magnetic field of the memory device. The computer-implemented method further includes, responsive to determining that at least one of the temperature and the magnetic field of the memory device reaches and / or exceeds a predetermined threshold, respectively, increasing the frequency at which data scrubbing is performed on the memory device.

Sanitizing data can be a cumbersome task, particularly when the volume of data is large, the content is sensitive, and / or the type of sanitation requires contextual determinations. Sanitizing large amounts of data is tedious and may often require highly trained personnel with clearances and / or other qualifications. In the systems and methods of the present disclosure, language models (LMs) are used to solve these and other technical issues with tools that may allow sanitizing data easily, with high versatility, context awareness, and / or low demand for computational resources. In particular, some of the disclosed systems and methods use a first language model and a second language model (being less resource-intensive than the first language model) to generate sanitized output data with improved efficiency and accuracy. This dual-model approach ensures that sensitive information is handled appropriately while optimizing computer resource usage.

The invention discloses a personal voice intelligent terminal for a fund transaction room. The personal voice intelligent terminal comprises a voice data acquisition and classification module, a data cleaning and structuring module, a task concurrency feature extraction module, a concurrency situation prediction module, a situation classification module, a normal situation processing module and a high concurrency situation processing module. And the voice data acquisition and classification module is used for acquiring voice data of traders in real time through terminal equipment in a financial transaction scene. According to the method, the voice data is collected and classified in real time, the feature engineering and the deep learning model are combined, the voice stream processing mode is dynamically adjusted, the high task concurrency situation is effectively dealt with, and misrecognition and resource conflicts are reduced. Voice streams are segmented and fragments are identified independently under the conditions of resource saving and high concurrency in a normal situation, so that major identification errors are prevented, fund loss and strategy failure risks are reduced, the market order stability is guaranteed, and the efficiency and security of a transaction room are improved.

The application discloses a ship carbon emission feature prediction method based on a dynamic method and an attention mechanism, adopts an AIS data-based dynamic method and a TFT time sequence prediction model based on the attention mechanism, utilizes ship AIS data, ship Lloyd's file data, pollutant emission factor parameters and geographic information, establishes a high-resolution ship carbon emission list, and realizes multi-period pollutant emission feature prediction of the ship. Through data cleaning, AIS data trajectory integration, data fusion and data completion and other operations, the AIS data is preprocessed, and the accuracy of the AIS data is ensured. According to the ship type and the ship running state, the dynamic method is used to estimate and analyze the pollutant emission amount, and the dynamic ship behavior is considered in real time when analyzing the carbon emission law and predicting the carbon emission feature.

The invention discloses an overlay-based traceless data cleaning method, which comprises the following steps of: after a file system is loaded when a system is started, creating an overlay file system, and overlapping and mounting contents of a root directory into the overlay file system according to a data protection partition range set by a user; data generated in the process of using the operating system by the user is stored in the overlay file system; and when data cleaning is needed, calling shred to perform physical layer deep cleaning on all data in the overlay file system. According to the method, the disk space does not need to be wasted for backup in advance, the file is backed up only when the file is changed, the disk space consumption is small (only the changed file is backed up), and the implementation (cleaning and restoring) time is short.

The invention discloses an airport parking apron target detection method based on Faster-RCNN, and belongs to the field of computer vision, and the method comprises the following steps: carrying out the continuous shooting of a parking apron region through an airport monitoring camera, and collecting image data, including various targets of airplanes, workers and ground service vehicles; processing the image data through methods of data cleaning, data enhancement, target labeling and the like; carrying out detailed statistical analysis on the processed data set to optimize a model training strategy; and optimization is carried out based on a Faster-RCNN target detection framework, and an airport parking apron target detection model is established. The target detection performance under the parking apron scene is improved, and therefore effective support is provided for airport operation management and safety monitoring.

The application relates to a method, device, equipment, storage medium and program product for estimating the position of an elevator. The method comprises: collecting an electrical signal of an elevator hoisting machine to obtain voltage and current in a target period; determining the voltage frequency and current frequency of the elevator hoisting machine according to the voltage and current respectively; calculating the angular velocity of the elevator hoisting machine according to the voltage frequency, current frequency and motor pole pair number, and determining the comprehensive angular velocity based on the voltage effective value corresponding to the voltage and the angular velocity; determining the start-stop time point of the elevator hoisting machine, and performing data cleaning on the comprehensive angular velocity according to the start-stop time point to obtain a target angular velocity; inputting the target angular velocity into an elevator system mathematical model to perform displacement calculation and obtain displacement information of an elevator car; and determining the position information of the elevator car based on the displacement information and the cumulative displacement information of the elevator car in the current uplink or downlink stage. The method can effectively improve the estimation accuracy of the position of the elevator.

An embodiment of the present application provides a container scheduling method, device, and storage medium. In an embodiment of the present application, during the container scheduling process, for the first container where the target cloud application that the first user requests to exit is located, a cleanup operation can be performed on the first container according to a preset cleanup instruction to clean up the container data generated by the first user using the first container, and the association between the first container and the first user can be released to obtain a restored first container. On this basis, the restored first container can be used for subsequent container scheduling. Accordingly, in an embodiment of the present application, during the container scheduling process, the container data left after the user uses the container is cleaned up and the association between the container and the user is released through a preset cleanup instruction, which can achieve pre-preparation of container resources and no restart after use, thereby effectively reducing resource overhead in the container scheduling process.

This invention discloses a YOLO-based method for detecting surface defects on printed circuit boards, belonging to the fields of computer vision and intelligent manufacturing technology. The method includes the following steps: deploying an industrial area scan camera at the inspection station of an automated production line; when a PCB enters the inspection station, the camera is triggered by a sensor to acquire and store images; the acquired image data is processed using data cleaning, data augmentation, and target annotation methods; the processed PCB defect dataset undergoes detailed statistical analysis to optimize the model training strategy; based on the data analysis results, optimization is performed using the YOLOv5 target detection framework to establish a surface defect detection model suitable for PCB production scenarios. This invention can effectively improve the automation level and detection accuracy of defect detection, is suitable for modern large-scale batch production of PCBs, and has good engineering application prospects and industrial promotion value.

The invention relates to a bridge technical condition rating prediction method and system fusing maintenance process information. The method comprises the following steps: constructing a bridge maintenance process information data set based on a contour coefficient method and a K-means algorithm, and constructing a bridge regular inspection information database; carrying out discretization processing on the numeric data after data cleaning, and converting the numeric data into classified data; calculating Cramer's V coefficients between all the classification data and the bridge technical condition rating information, based on the selected feature data, converting the data with classification data attributes into one-hot codes, and adopting a principal component analysis method to reduce dimensionality to obtain dimensionality-reduced classification data; performing normalization and equalization processing to obtain a data set for model training; training a bridge technical condition rating prediction model by using a PSO-LightGBM algorithm; and predicting a bridge technical state rating level by using the prediction model. According to the invention, the accuracy of bridge technical condition rating prediction is improved.

The invention provides an enterprise complex decision auxiliary method and system based on a multi-modal cognitive intelligent large model, and belongs to the technical field of artificial intelligence. The method comprises the steps of collecting multi-modal enterprise complex decision auxiliary data, and performing data cleaning and data labeling to obtain a training data set; constructing an enterprise complex decision-making auxiliary large model; inputting the training data set into the enterprise complex decision auxiliary large model, and optimizing large model training to obtain a trained enterprise complex decision auxiliary large model; and deploying the trained large model to a server, and generating decision suggestions in combination with an input data enhancement strategy. According to the method, through joint processing of the bimodal data, the reasonability and the intelligent degree of model decision can be remarkably improved, and the model prediction accuracy in different scenes is ensured.

The embodiment of the present application provides a data cleaning method, comprising: when data cleaning is needed for a target disk included in a target server, determining whether a target control card for controlling the disk is integrated in the target server; when the target control card is integrated in the target server and the target control card has a data cleaning function, performing a first data cleaning operation on the target disk by calling the data cleaning function of the target control card; when it is determined that the first data cleaning operation is completed, modifying a configuration mode of the target disk to a JBOD mode, restarting the target server and mounting a target boot image, wherein the target boot image includes a first tool supporting data cleaning of the target disk; determining a target tool for data cleaning of the target disk according to a kind of the target disk according to a predetermined rule, and performing a second data cleaning operation on the target disk by using the target tool, wherein the first tool includes the target tool.

The invention discloses an inactive user data management method, device and equipment and a storage medium, and relates to the technical field of data management, and the inactive user data management method comprises the following steps: configuring an inactive user data cleaning strategy, the data cleaning strategy defines a judgment condition of a user active state and at least one data cleaning operation corresponding to the judgment condition; periodically monitoring the active state of each user according to a preset time interval through the background service, and recognizing a target user meeting a data cleaning condition according to a judgment condition; and executing a data cleaning operation corresponding to the judgment condition on the target user to release the storage space. The problem of storage space waste of an Android operating system in a vehicle-mounted information entertainment system in a multi-user scene can be solved.

Embodiments of the present application provide a kind of processing method, device, electronic equipment, storage medium and product of abnormal data.The method comprises: obtaining the two-dimensional grid map to be processed;At least one specular reflection area is contained in the two-dimensional grid map;For any specular reflection area, determine a plurality of first line segments located in the specular reflection area, and determine the target boundary line segment of the specular reflection area based on the plurality of first line segments;Determine the spatial range of the specular reflection area based on the target boundary line segment, and delete all pixel points in the spatial range of the specular reflection area, provide a kind of systematic way to identify and remove abnormal data, reduce the need for manual intervention, support the data cleaning process of automation, and the spatial range of specular reflection area can be accurately determined, improve the cleaning efficiency and accuracy of abnormal data.

Embodiments of the present application provide a display memory control method, device, equipment, system and computer storage medium. The method comprises: in response to an obtained driving upgrade request of a graphics processing unit (GPU), obtaining a pre-upgrade page table corresponding to the GPU, the pre-upgrade page table comprising a mapping relationship between a physical address and a virtual address of GPU display memory; determining a data cleaning queue corresponding to the GPU, the data cleaning queue being used for globally cleaning data in the GPU display memory; under the condition of locking the data cleaning queue, performing a driving upgrade operation on the GPU based on the driving upgrade request, obtaining a post-upgrade page table corresponding to the GPU, the post-upgrade page table comprising a mapping relationship between the physical address and the virtual address of the GPU display memory; and based on the pre-upgrade page table and the post-upgrade page table, regulating the data in the GPU display memory. In the embodiments, the driving upgrade operation of the GPU can be implemented without the need for operations such as display memory backup, shutdown and restart.

The present disclosure describes methods, system, and devices for data processing in a mobile communication system. The method includes receiving, by a base station, data cleaning requirement for preprocessing measurement data from a first node providing control functions; receiving, by the base station, reference signal from a user equipment (UE); obtaining, by the base station, measurement data based on the reference signal; preprocessing, by the base station, the measurement data to obtain the preprocessed measurement data according to the data cleaning requirement; and sending, by the base station, the preprocessed measurement data to a second node for next step processing.

The invention provides an LDL-C concentration estimation method based on an interpretability enhancement machine, and belongs to the technical field of medical detection and artificial intelligence. The LDL-C concentration estimation method comprises the following steps: selecting a subject with a complete blood fat detection record from a preset database to obtain initial data; performing data cleaning on the initial data, and longitudinally merging the blood fat measurement data under the two visits to obtain a data set; performing regression modeling on the EBM based on the training set and the test set to obtain an EBM model; calculating a contribution value of each input ratio variable by constructing a square view of a lookup table based on an EBM model, adding all the contribution values, and calculating a final LDL-C predicted value through a connection function g; and performing comprehensive evaluation and optimization on the EBM model by adopting a plurality of statistical indexes, and integrating the EBM model into various medical systems. Through high-quality data, an optimized data processing strategy, an accurate EBM prediction model, an interpretable prediction result and efficient system integration, accurate estimation and wide application of LDL-C are realized, and the method is superior to a traditional formula calculation method.

The invention provides a hierarchical storage management method based on land observation satellite data value, which belongs to the field of application in a land observation satellite data processing system, and comprises the following steps of: calculating an initial comprehensive weight score during data archiving by creating seven types of core data tables such as a satellite parameter table and a data value parameter table; determining the number of storage days of the online or near-line cluster in combination with a dynamic storage strategy table; obtaining user order update data return times in real time, and recalculating the weight score at regular time to dynamically adjust the storage strategy; and finally, realizing data cleaning in combination with the water and fire wire threshold of the storage device. The problems of low resource utilization efficiency and slow user data return caused by making a storage strategy only based on the number of storage days in the prior art are solved, and the storage resource utilization rate and the user data response efficiency are remarkably improved.

The invention belongs to the technical field of ultrasonic transducers and artificial intelligence, and particularly discloses a piezoelectric transducer multi-target intelligent optimization design method which comprises the following steps: collecting performance data of a piezoelectric transducer, and performing data cleaning on the performance data; establishing a piezoelectric transducer model by using the processed data; and calculating the piezoelectric transducer model by using an NSGA-II multi-objective optimization algorithm to obtain optimal transducer design parameters, and completing the multi-objective intelligent optimization design of the piezoelectric transducer. According to the method, a more scientific and effective means is provided for design and optimization of the transducer by comprehensively applying a model establishment and multi-objective optimization method, and the problem that a large amount of time, manpower and resources need to be consumed by a manual experiment technology in the design of the piezoelectric transducer is solved.

The invention relates to the technical field of data cleaning, in particular to a data cleaning structure for CMOS data cleaning and a mainboard. The data cleaning structure comprises a body, an elastic piece and a trigger key; the body comprises fulcrums; the elastic piece has a set length, a first contact pin, a second contact pin and a third contact pin are arranged along the length direction of the elastic piece, a support position is arranged on the elastic piece between the second contact pin and the third contact pin, and the elastic piece is erected on the fulcrum through the support position; the trigger key comprises an abutting end, and the abutting end faces the third contact pin and is used for triggering or releasing triggering of the third contact pin; when the third contact pin is released from triggering, the first contact pin and the second contact pin are used for being in contact with a first contact piece and a second contact piece on the mainboard respectively; when the trigger key is triggered, the second contact pin and the third contact pin are used for being in contact with the second contact piece and the third contact piece on the mainboard respectively. The operation is simple and quick, the loss of parts can be avoided, and the situation of misoperation is reduced.

The invention relates to a line loss electric quantity analysis-based abnormal metering electric energy meter searching method, which comprises the following steps of: firstly, selecting a line of which the line loss rate exceeds a normal range, after confirming information of all node electric energy meters of the line, extracting daily frozen electric quantity readings and 15min frozen current data of all the electric energy meters on the line within a period of time, and performing data cleaning; calculating daily line loss electric quantity of the line and a daily current summation value of each node electric energy meter on the line by using the cleaned data, identifying and deleting an abnormal value according to a Pearson criterion, and then calculating a pearson correlation coefficient between the line loss electric quantity of the line and the summation current of each node electric energy meter; when the correlation coefficient between the current summation value of the node n electric energy meter and the line loss electric quantity is larger than 0.7, it can be judged that strong correlation exists between the current summation value of the node n electric energy meter and the overproof line loss electric quantity, and it is judged that metering abnormity exists in the node n electric energy meter according to the strong correlation. According to the invention, the capability and the level of metering abnormity online detection are effectively improved.

A data lifecycle management method, device, equipment, storage medium and program product are disclosed, access heat of all data of a first storage layer in a data warehouse is acquired; wherein the data warehouse comprises at least two storage layers, and there is an association relationship between the storage layers, in which upstream data supports downstream data; cold data in the first storage layer is determined based on the access heat; associated data related to the cold data in the remaining storage layers is determined based on the association relationship; and a data cleaning operation is performed on the cold data and the associated data. By adopting the embodiment of the present application, the problems of hot data misdeletion and cold data redundancy in the data warehouse can be avoided, fine and differentiated management of data is realized, and resource waste of the data warehouse is reduced.

The invention discloses a marine environment element prediction method based on a principal component analysis-converter fusion model. The method comprises the following steps: performing quality control and data cleaning on a marine environment element data set to realize time-space dimension unification of heterogeneous data; building a prediction model on the basis of a converter network, taking day-by-day data after quality control and data cleaning as a target variable of the model, performing dynamics mechanism analysis based on an observation time sequence of the target variable, and selecting a dynamics prediction factor; dimensionality reduction processing is carried out on the high-dimensional forecasting factors through principal component analysis, and key feature information is effectively extracted; and inputting the optimized features into a converter network for modeling prediction. Compared with a traditional converter model, the average prediction relative error of the fusion model provided by the invention is reduced by 30%, and the prediction precision is remarkably improved. According to the invention, a new thought is provided for improving the marine environment element prediction precision through deep fusion, and more reliable decision support can be provided for the fields of marine navigation safety and the like.