44 results about "Data scrubbing" patented technology

The invention relates to a system and method for low-voltage judgment of a power failure area based on mobile optical modem equipment, and the method comprises the steps: collecting the static data and offline event data of each mobile optical modem in real time, and storing the data after the data cleaning operation; based on historical offline event data of each mobile optical modem, an initial weight is distributed for each optical modem offline event, and then the offline weight of each mobile optical modem is dynamically adjusted according to the spatial and temporal distribution characteristics of the real-time offline event data of each mobile optical modem; in combination with the offline weight of each mobile optical modem, determining the power failure judgment weight of the corresponding area through space-time scanning statistics; and comparing the power failure judgment weight of each area with a corresponding power failure threshold value to judge whether a power failure fault occurs in the area, and if the power failure fault occurs, outputting position information and power failure warning information corresponding to the area. Compared with the prior art, the method can timely and accurately identify the regional power failure event and determine the fault region.

The present disclosure provides a method, apparatus and system for monitoring ultra-high frequency partial discharge of a hydro-generator, and belongs to the technical field of hydro-generator partial discharge monitoring. The method includes: cleaning a partial discharge pulse sequence using a cleaning threshold to obtain a valid pulse sequence; performing redundant data filtering on each data unit divided from the valid pulse sequence to obtain a first target pulse sequence for short-period partial discharge monitoring; determining sub-sequences that are partial discharge events from the valid pulse sequence, forming a second target pulse sequence after associating an amplitude statistical feature of the sub-sequences, and storing the second target pulse sequence. The aforementioned method combines data cleaning, redundant data filtering, and partial discharge event identification, which enhances the real-time performance of partial discharge monitoring and records the long-period partial discharge change trend. The corresponding system adopts a multi-buffer zone and multi-processor architecture, thus further improving the real-time performance of partial discharge monitoring.

The embodiment of the invention provides a video memory control method, device, equipment and system and a computer storage medium. The method comprises the steps that in response to an obtained drive upgrading request of an image processing unit (GPU), a before-upgrading page table corresponding to the GPU is obtained, and the before-upgrading page table comprises a mapping relation between a physical address and a virtual address of a GPU video memory; determining a data cleaning queue corresponding to the GPU, wherein the data cleaning queue is used for performing global cleaning on data in a GPU video memory; under the condition that the data cleaning queue is locked, driving upgrading operation is carried out on the GPU based on the driving upgrading request, an upgraded page table corresponding to the GPU is obtained, and the upgraded page table comprises a mapping relation between a physical address and a virtual address of a GPU video memory; and regulating and controlling data in the GPU video memory based on the pre-upgrading page table and the post-upgrading page table. In the embodiment of the invention, the driver upgrading operation of the GPU can be realized without operations such as video memory backup, shutdown, restart and the like.

The application discloses a ship carbon emission feature prediction method based on a dynamic method and an attention mechanism, adopts an AIS data-based dynamic method and a TFT time sequence prediction model based on the attention mechanism, utilizes ship AIS data, ship Lloyd's file data, pollutant emission factor parameters and geographic information, establishes a high-resolution ship carbon emission list, and realizes multi-period pollutant emission feature prediction of the ship. Through data cleaning, AIS data trajectory integration, data fusion and data completion and other operations, the AIS data is preprocessed, and the accuracy of the AIS data is ensured. According to the ship type and the ship running state, the dynamic method is used to estimate and analyze the pollutant emission amount, and the dynamic ship behavior is considered in real time when analyzing the carbon emission law and predicting the carbon emission feature.

The invention discloses an airport parking apron target detection method based on Faster-RCNN, and belongs to the field of computer vision, and the method comprises the following steps: carrying out the continuous shooting of a parking apron region through an airport monitoring camera, and collecting image data, including various targets of airplanes, workers and ground service vehicles; processing the image data through methods of data cleaning, data enhancement, target labeling and the like; carrying out detailed statistical analysis on the processed data set to optimize a model training strategy; and optimization is carried out based on a Faster-RCNN target detection framework, and an airport parking apron target detection model is established. The target detection performance under the parking apron scene is improved, and therefore effective support is provided for airport operation management and safety monitoring.

The application relates to a method, device, equipment, storage medium and program product for estimating the position of an elevator. The method comprises: collecting an electrical signal of an elevator hoisting machine to obtain voltage and current in a target period; determining the voltage frequency and current frequency of the elevator hoisting machine according to the voltage and current respectively; calculating the angular velocity of the elevator hoisting machine according to the voltage frequency, current frequency and motor pole pair number, and determining the comprehensive angular velocity based on the voltage effective value corresponding to the voltage and the angular velocity; determining the start-stop time point of the elevator hoisting machine, and performing data cleaning on the comprehensive angular velocity according to the start-stop time point to obtain a target angular velocity; inputting the target angular velocity into an elevator system mathematical model to perform displacement calculation and obtain displacement information of an elevator car; and determining the position information of the elevator car based on the displacement information and the cumulative displacement information of the elevator car in the current uplink or downlink stage. The method can effectively improve the estimation accuracy of the position of the elevator.

This invention discloses a YOLO-based method for detecting surface defects on printed circuit boards, belonging to the fields of computer vision and intelligent manufacturing technology. The method includes the following steps: deploying an industrial area scan camera at the inspection station of an automated production line; when a PCB enters the inspection station, the camera is triggered by a sensor to acquire and store images; the acquired image data is processed using data cleaning, data augmentation, and target annotation methods; the processed PCB defect dataset undergoes detailed statistical analysis to optimize the model training strategy; based on the data analysis results, optimization is performed using the YOLOv5 target detection framework to establish a surface defect detection model suitable for PCB production scenarios. This invention can effectively improve the automation level and detection accuracy of defect detection, is suitable for modern large-scale batch production of PCBs, and has good engineering application prospects and industrial promotion value.

The invention relates to a bridge technical condition rating prediction method and system fusing maintenance process information. The method comprises the following steps: constructing a bridge maintenance process information data set based on a contour coefficient method and a K-means algorithm, and constructing a bridge regular inspection information database; carrying out discretization processing on the numeric data after data cleaning, and converting the numeric data into classified data; calculating Cramer's V coefficients between all the classification data and the bridge technical condition rating information, based on the selected feature data, converting the data with classification data attributes into one-hot codes, and adopting a principal component analysis method to reduce dimensionality to obtain dimensionality-reduced classification data; performing normalization and equalization processing to obtain a data set for model training; training a bridge technical condition rating prediction model by using a PSO-LightGBM algorithm; and predicting a bridge technical state rating level by using the prediction model. According to the invention, the accuracy of bridge technical condition rating prediction is improved.

The invention discloses an inactive user data management method, device and equipment and a storage medium, and relates to the technical field of data management, and the inactive user data management method comprises the following steps: configuring an inactive user data cleaning strategy, the data cleaning strategy defines a judgment condition of a user active state and at least one data cleaning operation corresponding to the judgment condition; periodically monitoring the active state of each user according to a preset time interval through the background service, and recognizing a target user meeting a data cleaning condition according to a judgment condition; and executing a data cleaning operation corresponding to the judgment condition on the target user to release the storage space. The problem of storage space waste of an Android operating system in a vehicle-mounted information entertainment system in a multi-user scene can be solved.

Embodiments of the present application provide a kind of processing method, device, electronic equipment, storage medium and product of abnormal data.The method comprises: obtaining the two-dimensional grid map to be processed;At least one specular reflection area is contained in the two-dimensional grid map;For any specular reflection area, determine a plurality of first line segments located in the specular reflection area, and determine the target boundary line segment of the specular reflection area based on the plurality of first line segments;Determine the spatial range of the specular reflection area based on the target boundary line segment, and delete all pixel points in the spatial range of the specular reflection area, provide a kind of systematic way to identify and remove abnormal data, reduce the need for manual intervention, support the data cleaning process of automation, and the spatial range of specular reflection area can be accurately determined, improve the cleaning efficiency and accuracy of abnormal data.

Embodiments of the present application provide a display memory control method, device, equipment, system and computer storage medium. The method comprises: in response to an obtained driving upgrade request of a graphics processing unit (GPU), obtaining a pre-upgrade page table corresponding to the GPU, the pre-upgrade page table comprising a mapping relationship between a physical address and a virtual address of GPU display memory; determining a data cleaning queue corresponding to the GPU, the data cleaning queue being used for globally cleaning data in the GPU display memory; under the condition of locking the data cleaning queue, performing a driving upgrade operation on the GPU based on the driving upgrade request, obtaining a post-upgrade page table corresponding to the GPU, the post-upgrade page table comprising a mapping relationship between the physical address and the virtual address of the GPU display memory; and based on the pre-upgrade page table and the post-upgrade page table, regulating the data in the GPU display memory. In the embodiments, the driving upgrade operation of the GPU can be implemented without the need for operations such as display memory backup, shutdown and restart.

The invention provides a hierarchical storage management method based on land observation satellite data value, which belongs to the field of application in a land observation satellite data processing system, and comprises the following steps of: calculating an initial comprehensive weight score during data archiving by creating seven types of core data tables such as a satellite parameter table and a data value parameter table; determining the number of storage days of the online or near-line cluster in combination with a dynamic storage strategy table; obtaining user order update data return times in real time, and recalculating the weight score at regular time to dynamically adjust the storage strategy; and finally, realizing data cleaning in combination with the water and fire wire threshold of the storage device. The problems of low resource utilization efficiency and slow user data return caused by making a storage strategy only based on the number of storage days in the prior art are solved, and the storage resource utilization rate and the user data response efficiency are remarkably improved.

The invention relates to the technical field of data cleaning, in particular to a data cleaning structure for CMOS data cleaning and a mainboard. The data cleaning structure comprises a body, an elastic piece and a trigger key; the body comprises fulcrums; the elastic piece has a set length, a first contact pin, a second contact pin and a third contact pin are arranged along the length direction of the elastic piece, a support position is arranged on the elastic piece between the second contact pin and the third contact pin, and the elastic piece is erected on the fulcrum through the support position; the trigger key comprises an abutting end, and the abutting end faces the third contact pin and is used for triggering or releasing triggering of the third contact pin; when the third contact pin is released from triggering, the first contact pin and the second contact pin are used for being in contact with a first contact piece and a second contact piece on the mainboard respectively; when the trigger key is triggered, the second contact pin and the third contact pin are used for being in contact with the second contact piece and the third contact piece on the mainboard respectively. The operation is simple and quick, the loss of parts can be avoided, and the situation of misoperation is reduced.

A data lifecycle management method, device, equipment, storage medium and program product are disclosed, access heat of all data of a first storage layer in a data warehouse is acquired; wherein the data warehouse comprises at least two storage layers, and there is an association relationship between the storage layers, in which upstream data supports downstream data; cold data in the first storage layer is determined based on the access heat; associated data related to the cold data in the remaining storage layers is determined based on the association relationship; and a data cleaning operation is performed on the cold data and the associated data. By adopting the embodiment of the present application, the problems of hot data misdeletion and cold data redundancy in the data warehouse can be avoided, fine and differentiated management of data is realized, and resource waste of the data warehouse is reduced.

The application discloses a Redis cache elimination middleware based on a cuckoo filter, which is arranged between Redis and an application layer and comprises the following modules: a hot degree tracking module, a cold data cleaning module, an accurate elimination module, an adaptive tuning module, a memory management module and a concurrent control module. The application can realize intelligent elimination and dynamic management of a Redis cache space, and through accurate hot degree tracking, distributed cleaning, accurate elimination, adaptive tuning, intelligent memory management and efficient concurrent control, can effectively solve problems of low cache space utilization, long-term occupation of space by cold data and low elimination accuracy caused by a traditional cache elimination strategy, and improve the overall efficiency and resource utilization of a cache system.

There is provided a technique that contributes to the optimization of data cleaning prior to data analysis. A method for optimizing data processing of an input table includes performing a limitation process in which an extraction table is generated from a first target table using bits of a null bitmap by extracting a column having one or more missing values, the null bitmap being a metadata of the first target table; and performing a count process in the extraction table, in the count process the number of missing values per row of the extraction table being obtained by performing a calculation with the bits of the null bitmap, wherein, the limitation process is performed with the input table as the first target table.

The present disclosure relates to a bridge technical condition rating prediction method and system fusing maintenance history information. The method comprises: constructing a bridge maintenance history information dataset based on the profile coefficient method and the K-means algorithm, and constructing a bridge inspection information database; discretizing the numerical data after data cleaning to convert it into classification data; calculating the Cramer's V coefficient between all classification data and bridge technical condition rating information, selecting feature data based on this, converting data with classification data attributes into one-hot encoding, and reducing the dimension using the principal component analysis method to obtain reduced classification data; normalizing and equalizing to obtain a dataset for model training; training a bridge technical condition rating prediction model using the PSO-LightGBM algorithm; and predicting the bridge technical state rating level using the prediction model. The present disclosure improves the accuracy of bridge technical condition rating prediction.

The application discloses a high-dimensional soil data visualization method and system based on density peak clustering, and the visualization method comprises the following steps: S1, a data set comprising a plurality of samples is acquired, and data cleaning and standardization processing are performed on the samples in the data set, and each sample comprises various types of rock-soil data; S2, the data set after the standardization processing is labeled according to the geographical positions of stations, and then dimension reduction processing is performed on the data set after the label labeling, so as to obtain 3-dimensional reduced data; S3, a density peak clustering algorithm is used to cluster the 3-dimensional reduced data, so as to obtain a clustering result; and S4, the visualization method is used to convert the clustering result into a graphic display.

The application discloses a data cleaning method and device, equipment, medium and product, comprising: displaying a data cleaning configuration control and M data table names corresponding to a first user identifier in a first configuration interface; receiving a first selection operation of selecting a first data table name and a second selection operation of the data cleaning configuration control in the first configuration interface; in response to the first selection operation and the second selection operation, displaying N fields in the first data table, and receiving a third selection operation of selecting a first field; in response to the third selection operation, determining that the first field is a cleaning basis field, and dividing the first data table into each data partition based on the cleaning basis field; displaying data distribution of each data partition in a second configuration interface, and receiving a cleaning range configuration operation in the second configuration interface; determining a cleaning range based on the cleaning range configuration operation, determining data in the first data table located in the cleaning range as to-be-cleaned data, and cleaning the to-be-cleaned data.

The invention relates to the technical field of traffic management, in particular to a method for etching a traffic management level based on alarm data of alarm conditions, police forces and road abnormal events, which comprises the following steps of: acquiring alarm condition data, police force data, traffic abnormal event data and the total number of motor vehicles from corresponding data sources through a supercomputing center; after data cleaning, data comparison, data conversion and data analysis, a traffic abnormal event factor index, an alarm condition factor index, a police force factor index, a traffic abnormal event and police force matching degree index, an alarm condition and police force matching degree index and a police force execution efficiency index are calculated according to the traffic management level description model; calculating a traffic management comprehensive index according to each index and the corresponding weight; and according to the score range of the traffic management comprehensive index, performing grade evaluation on the traffic management level and generating a corresponding police force allocation scheme. The system can accurately evaluate the traffic management level and generate a police force allocation scheme, and efficiently optimizes the traffic management.

The invention relates to the technical field of data preprocessing, and provides a double-door sentry type entrance dirty data cleaning and error classification method and system. The double-door sentry type entrance dirty data cleaning and error classification method comprises the following steps: calculating the root mean square of each window; calculating spectrum dispersion; if the minimum value of the root-mean-square is smaller than the idle calibration value of the production line or the spectrum dispersion is larger than a spectrum dispersion threshold value, the event window is discarded; calculating a vibration peak value or a root-mean-square of the offline industrial sensing signal data, and if the vibration peak value or the root-mean-square exceeds a first threshold value, determining that the data is signal distortion data; performing consistency judgment on the sampling rate of the off-line industrial sensing signal data, and screening sampling rate error data; and calculating a cross correlation coefficient and an energy difference between the current event window and the previous frame event window, and if the cross correlation coefficient is smaller than a second threshold value and the energy difference is more than K times, judging that the data is packet loss data. The closed loop of training data self-cleaning and error reutilization in the offline environment of the local area network is realized.

The embodiment of the application provides a sandbox-based data cleaning method and system, electronic equipment and storage medium, and relates to the technical field of computers. The method is applied to a set application program in a user terminal, the set application program is in communication connection with a target application program running in a sandbox mechanism, and the method comprises the following steps: establishing a communication connection with a server; receiving a data cleaning instruction sent by the server, wherein the data cleaning instruction comprises the identification of the target application program and a to-be-cleaned data storage path, wherein the server sends the data cleaning instruction based on the cache data amount under each data storage path in the sandbox corresponding to the target application program acquired in real time; and sending the data cleaning instruction to the target application program based on the identification of the target application program, so that the target application program cleans the cache data under the to-be-cleaned data storage path. The application does not need the user to manually initiate the data cleaning instruction, and the cleaning efficiency and accuracy are relatively high.

The application discloses a kind of non-active user data management method, device and equipment and storage medium, it is related to data management technical field, non-active user data management method includes: configuration non-active user data cleaning strategy, data cleaning strategy defines the determination condition of user active state and at least one data cleaning operation corresponding to determination condition;Periodically monitor the active state of each user according to preset time interval by background service, according to determination condition, identify target user that satisfies data cleaning condition;The data cleaning operation corresponding to determination condition is executed to target user, to release storage space.The application can solve the waste problem of storage space in the multi-user scene of Android operating system in vehicle information entertainment system.

A method to perform data scrub operations by operating a memory device, the memory device comprising a main memory, an internal Error Correction Code (ECC) engine and a scrub memory. The method comprising: receiving a read command; accessing, based on the receiving the read command, a location in the main memory to read data at the location; error correcting data read during the accessing; and storing at the scrub memory information of the location based at least in part on the correcting the data meeting or exceeding an ECC threshold.

The application discloses an on-vehicle data cleaning method and device, electronic equipment and a storage medium, and aims at solving the technical problem of poor quality of the existing on-vehicle data cleaning method. The application comprises the following steps: reading a vehicle state log file; generating a filter table by using the automatic driving state filter rule table and the vehicle state log file; obtaining a bag file; filtering the bag file by using the filter table to obtain a target bag file, and deleting the bag file.

The invention relates to the technical field of industrial automation and intelligent control, and discloses a self-adaptive intelligent cleaning control method and system for a train open wagon, and the method comprises the steps: synchronously obtaining a material apparent image and tool-material interaction data when a target point is cleaned; obtaining an effect image after cleaning, calculating a residual rate, and deciding whether to reclean according to the residual rate; extracting and fusing based on the image and mechanical data to generate a multi-modal dynamic state feature vector; taking the feature vector as an input, taking an actual control parameter and a cleaning effect evaluation parameter as an output, constructing and continuously updating a time-varying inverse model on line, and adopting a model prediction control algorithm to solve an optimal control parameter vector of a next target point to be cleaned in a rolling manner; finally, the cleaning device is driven to execute self-adaptive cleaning, and a new round of control circulation is started. The cleaning device can automatically adapt to spatial changes of material characteristics, and intelligent global optimization of the cleaning effect, efficiency and energy consumption is achieved.

The present application provides a batch processing method for fiber static tensile test data, comprising: data reading and data processing, stress-strain data extraction, Young's modulus related calculation fitting point screening, linear fitting, data point marking and point graph output, data cleaning and origin processing, breaking strength, elongation and breaking work calculation, unit calibration and single group data output, cyclic processing and multi-group data operation, single group data Excel output, single Excel statistics and total Excel output. The present application solves the problem of lack of batch and accurate automatic processing method for fiber tensile machine data, significantly improves the efficiency and accuracy of processing batch data, is simple to operate, can be applied to various fiber tensile stress-strain data processing, and through the output of tensile stress-strain point graph, the data accuracy can be observed intuitively, effectively helps people to reduce the time cost consumption in the process of processing data, and provides a more efficient, accurate and user-friendly solution for fiber tensile test data processing.

The invention discloses a severe environment-oriented island power distribution network multi-modal data fusion and early warning method, which comprises the steps of obtaining multi-modal data in an island power distribution network operation process and performing data preprocessing, including data cleaning, de-noising and normalization processing, so as to obtain multi-modal normalized data of the same dimension; extracting equipment image features and equipment electrical operation features from the multi-modal normalized data, performing weighted fusion processing to obtain a fusion feature matrix, and performing equipment state recognition on the fusion feature matrix through an intelligent detection model; and when a fault condition is met, identifying a fault type and positioning a fault position to obtain a fault detection result of the island power distribution network equipment, carrying out fault early warning strategy matching of a corresponding risk level on the island power distribution network equipment according to the fault detection result, and executing the matched fault early warning strategy to carry out fault early warning. The method has the advantages of improving the accuracy and response speed of fault detection and effectively guaranteeing safe and stable operation of the island power distribution network.

The invention provides an aircraft key component target detection method based on YOLOv5, and belongs to the field of computer vision. Comprising the following steps: step 1; an airport monitoring camera is used for continuously shooting a parking apron area, image data are collected, and collected images comprise an aircraft target; 2, for the collected image data, processing the image data through methods such as data cleaning, data enhancement and target labeling; 3, carrying out detailed statistical analysis on the processed data set to optimize a model training strategy; and 4, aiming at a data analysis result, performing optimization based on a YOLOv5 target detection framework, and establishing a target detection model suitable for the key components of the aircraft. The method can be widely applied to the fields of aviation safety monitoring, aircraft maintenance and detection, airport ground guarantee and the like, and an efficient and accurate target detection solution is provided for the aerospace industry.