51 results about "Data balancing" patented technology

The invention discloses a femoral head necrosis prediction system and method based on Transform, and belongs to the technical field of image processing, the system comprises a data acquisition module, a data processing module, a data balancing module, a feature extraction module, a feature merging module and a necrosis classification module; the method comprises the following steps: acquiring preoperative hip joint CT data, pre-processing to unify specifications, adopting directional data enhancement to solve the problem of class imbalance, extracting features by virtue of Video Swin Transform, merging and modeling a spatial relationship through volume blocks, and finally outputting a necrosis classification result. According to the femoral head necrosis prediction system and method based on Transform provided by the invention, the prediction accuracy and robustness are effectively improved, the problem of model instability caused by strong subjectivity, hysteresis and data imbalance in the prior art is solved, an objective and efficient prediction tool is provided for clinic, and the risk of postoperative complications is reduced.

The invention discloses a training data balance processing method and device, equipment, a medium and a product. The invention relates to the technical field of data processing. The method comprises the following steps: when a user in a user set is associated with an object in an object set, generating associated data according to the associated user and object, and adding the associated data into an associated data set; acquiring other users of the same category as the target user corresponding to the to-be-supplemented data of the associated data set; obtaining a target object associated with the target user in the associated data set; according to each target object, determining candidate objects of the target user in other objects associated with the other users in the associated data set; and generating associated data according to the target user and each candidate object, and adding the associated data into the associated data set. The embodiment of the invention can balance the training data of the model.

The embodiment of the invention provides a data balancing method, device and equipment of a Ceph cluster and a medium, a plurality of placement groups are deployed in the Ceph cluster, each placement group comprises a master copy and a slave copy, the master copy and the slave copy of each placement group are placed on different storage devices in the Ceph cluster, and the method comprises the steps that data distribution information of the Ceph cluster is acquired; according to the data distribution information, performing a first balance operation on a primary copy of a placement group borne on each storage device in the Ceph cluster; and according to the data distribution information, carrying out second balance operation on the slave copies of the placement groups borne on the storage devices in the Ceph cluster. According to the embodiment of the invention, the code of the Ceph does not need to be modified, the number of the slave copies is balanced after the number of the master copies is balanced according to the data distribution information of the Ceph cluster, and the overall performance and the resource utilization rate of the cluster are improved and the risk caused by upgrading of the old version Ceph cluster is also avoided by ensuring the double-sided balance of the placement group in the Ceph cluster.

The invention belongs to the technical field of breast cancer molecular typing, and more specifically relates to a mammary gland ultrasound video HER2 expression state identification method based on spatial-temporal feature interaction. The method comprises the following steps: taking an original breast ultrasonic video as input data, performing multi-stage data preprocessing operation, and then performing data balance on training data to obtain a standardized training set; sequentially constructing a UniFormerV2 network-based feature module, a multi-stage feature fusion module and a classification module, and training to obtain a trained model; and inputting the test set into the model to obtain a final prediction result of the HER2 expression state. The problems that an existing HER2 detection method is invasive in operation and cannot perform dynamic monitoring, and an existing deep learning method based on static images neglects space-time dynamic information in an ultrasonic video, so that prediction accuracy of the HER2 expression state, especially HER2-Low subtype is insufficient are solved.

The application discloses a data balancing method and device, equipment and storage medium, and relates to the technical field of computers. The method comprises the following steps: performing hash processing on first samples and second samples in a data set, and mapping the first samples and the second samples into first hash buckets and second hash buckets; determining second samples matched with the first samples in the first hash buckets from second hash buckets adjacent to first hash buckets in the first hash buckets; searching in adjacent hash buckets in a counterfactual search process, reducing a search range, reducing a search amount, and improving search efficiency. Then, based on a feature relationship between the first samples and the second samples, third samples in the second samples are processed to obtain fourth samples. The labels of the fourth samples are consistent with the labels of the first samples, and data balancing is realized. Since the time required by the search process is shortened, the efficiency of data balancing is improved.

This application discloses a method for scaling up nodes in a distributed search engine, including obtaining the current information of the distributed search engine cluster and the information of the scaling nodes; calculating the total number M of shards that the scaling nodes can accommodate based on the current information of the cluster and the information of the scaling nodes; adding the scaling nodes to the cluster; performing multiple rounds of node pre-allocation, each round of node pre-allocation including: establishing a weight index and dividing the weight index into N weight shards; performing data balancing on the cluster; and completing node allocation when N reaches a preset value. The solution provided by this application solves the problem in the prior art where distributed search engines cannot scale up the cluster in-situ or on demand when the cluster data capacity or data write performance reaches its limit, resulting in resource waste.

The application relates to an accident prediction method and system fusing time mixing effect and machine learning, wherein the method comprises the following steps: acquiring traffic state data containing a time category and traffic accident data, and dividing the traffic state data into accident data and non-accident data based on the traffic accident data, taking the accident data as a training data set of a data balancing model; generating balanced accident data by using the trained data balancing model, integrating the accident data, the balanced accident data and a non-accident data set to obtain a first data set, and taking the first data set after pretreatment as a training data set of a real-time accident prediction model; inputting real-time collected traffic state data into the trained real-time accident prediction model to output an accident prediction result; and the system is used for realizing the above method. Compared with the prior art, the data quality generated by the sampling method is guaranteed through data balancing processing, and the prediction result of the accident is more accurate.

The application relates to a power grid edge device instruction set data balancing method based on a federal learning framework, which comprises the following steps: collecting instruction information D of power grid edge devices and the type L to which the instruction belongs for a power grid system in n regions; constructing a feature vector matrix of the device instruction information; initializing a conditional generative adversarial network model on a server, generating an RSA public and private key pair on the server, sending the public key to the power grid system in each region, and sending the model to the power grid system in each region; training the conditional generative adversarial network using the instruction data set and the label set collected by the region; until the power grid system in each of the n regions completes the training of the conditional generative adversarial network model by using the data set; performing federal scheduling through the server; and forming a large-scale balanced data set. The application uses the large-scale balanced data set to train the power grid edge device instruction anomaly detection model of each region, and accurately evaluates the device security vulnerabilities.

A system and method for filtered visualization of dynamic load monitoring via a power distribution unit (PDU) collects power measurement data measured from each end device plugged into, and drawing operating power from, the PDU. The PDU collects the power measurement data into power state data over a line cycle or series of time intervals. Within the PDU, Kalman filtering determines a current power state estimate based on the measured power state data balanced with predictions of subsequent power state, as well as tuning parameters defining the balance from line cycle to line cycle. The resulting power state estimate may be displayed or presented to provide a more intuitive representation of operating power drawn by the end devices as opposed to raw power measurement data, which may be subject to fluctuation caused by high performance / AI pulsing loads.

The application belongs to the field of biomedical signal processing and human-computer interaction, and discloses a muscle-derived signal gesture recognition method and system based on crown porcupine optimization VMD parameters. The method collects electromyography or muscle magnetic signals during gesture action, and constructs a sample set through sliding window segmentation, data balancing processing and standardization. The crown porcupine optimization algorithm (CPO) is used to adaptively optimize the mode number and penalty factor of variational mode decomposition (VMD), and the gesture classification accuracy is used as the fitness function. Global search and local mining are realized through the four-layer defense of vision, sound, odor and physics of CPO. The intrinsic mode function (IMF) under the optimal parameters is extracted and spliced into a multi-dimensional feature matrix, which is input into a classification model to realize gesture recognition. The application directly optimizes the recognition performance, enhances the feature discriminability, and improves the recognition accuracy and robustness of muscle-derived signals in complex scenes.

The invention discloses a data redistribution method, device and equipment for a distributed memory database and a medium, and belongs to the field of data migration. The method comprises the steps that expansion nodes are added to the distributed memory database, and data balance processing is conducted on the distributed memory database; screening target migration data from the to-be-migrated data according to the transmission time consumption calculated in real time; and performing dynamic migration processing according to the target migration data. According to the method, the data volume of each migration is determined through the predetermined acceptable service pause duration, the data can be divided into smaller data which can be flexibly split for transmission, and the granularity of the data is refined; dynamic adjustment is carried out according to the real-time migration state, resource competition with foreground transactions is reduced, resource waste is avoided, and the resource utilization rate is increased.

The invention discloses a log association judgment method and system based on TF-IDF and data balance. According to the method, a unified corpus is constructed through preprocessing, then the preprocessed BUG overview and description are spliced to serve as a query text, log fragments serve as documents, and a feature matrix is generated through TF-IDF vectorization. And further calculating the similarity between the query text and the document, and generating a final feature vector in combination with length weighting and position weighting. The problem of data imbalance is solved through undersampling or oversampling processing, and model training is carried out by using logistic regression or a linear SVM (Support Vector Machine). And finally, performing storage-friendly persistence processing on the trained model, and performing log association judgment on the processed model. According to the method, the accuracy of correlation judgment between the Chinese BUG report and the DMSG log fragment is remarkably improved, the misjudgment rate is reduced, meanwhile, the efficiency of model training and deployment is improved, and the generalization ability and engineering adaptability of the model are enhanced.

This invention belongs to the technical field of breast cancer molecular subtyping, and more specifically, relates to a method for identifying HER2 expression status in breast ultrasound videos based on spatiotemporal feature interaction. The method includes: using raw breast ultrasound videos as input data and performing multi-level data preprocessing operations; then performing data balancing on the training data to obtain a standardized training set; sequentially constructing a feature module, a multi-stage feature fusion module, and a classification module based on the UniFormerV2 network, and training the resulting model; and inputting the test set into the model to obtain the final prediction result of HER2 expression status. This invention solves the problems of existing HER2 detection methods being invasive and unable to dynamically monitor, and existing deep learning methods based on static images ignoring the spatiotemporal dynamic information in ultrasound videos, leading to insufficient accuracy in predicting HER2 expression status, especially the HER2-Low subtype.

The invention discloses a personalized health management scheme generation method based on artificial intelligence, and relates to the technical field of artificial intelligence and health management, and the method comprises the following steps: constructing a multi-modal, cross-time and cross-data-source data integration platform, the method comprises the following steps: gathering multi-source health data from drugstores in different geographic areas, cross-time sequence data and a multi-source heterogeneous data set of a third-party data source, carrying out preprocessing of cleaning, format unification and feature coding, and pre-evaluating data skewness by adopting a statistical analysis method; according to the method, multi-mode, cross-time and third-party data skew characteristics are comprehensively captured, multi-source heterogeneous data integration and skew pre-evaluation are carried out, and third-party data source skew transmission correction and data balance are carried out. The cooperative influence of multi-modal data skew, the cumulative effect of data cross-time skew and the transmission problem of third-party data source skew are effectively solved, and the accuracy and balance of data are improved.

The invention discloses an online game dialogue malicious language interpretable detection method based on knowledge distillation, and belongs to the field of natural language processing and artificial intelligence. According to the method, interpretable labels are generated through multi-stage labeling, T5 model semantic retelling is used for balancing data distribution, a combined training framework of a classifier and a reason generator is combined, and efficient interpretable harmfulness detection and dynamic enhancement of large-scale game dialogues are achieved based on focus loss function optimization and manual consistency verification. According to the method, high-precision interpretable detection is achieved through the F1 score of the classifier and the BLEU-4 score of the reason generator, the calculation cost is reduced in combination with knowledge distillation, the low-frequency malicious recognition robustness is enhanced through a data balance strategy, and triple improvement of high efficiency, transparency and adaptability of large-scale game dialogue harmfulness analysis is achieved.

The invention discloses a text data balancing method based on adversarial training, and belongs to the field of data balancing, and the method comprises the steps: determining a target text sub-data set from each text sub-data set of a text data set to be subjected to data balancing; establishing a text generation cooperation strategy corresponding to the target text sub-data set based on the target text sub-data set, and performing cooperation adjustment on a text generator and a generated text discriminator respectively corresponding to the target text sub-data set based on the text generation cooperation strategy; and performing data expansion on the target text sub-data set based on the adjusted text generator to obtain a text data set after data balance. According to the method, a collaborative adjustment mechanism based on an adversarial thought is introduced, so that the balance between data diversity and simulation degree is realized while the category distribution deviation is effectively eliminated, and the problem that the data balance and the data capacity cannot be considered at the same time is solved.

The invention relates to the technical field of machine learning, and provides a prompt generation method and device, computer equipment and a readable storage medium, and the method comprises the steps: obtaining business data, rule data and historical processing records, and carrying out the preprocessing, and forming a standardized data set; obtaining a rule matrix corresponding to the standardized data set in real time by adopting increment extraction; performing dimension reduction and distribution equalization on the high-dimensional features corresponding to the rule matrix by using correlation analysis, a feature selection algorithm and a data balance technology; through loss distribution fitting, cross validation and hyper-parameter optimization, constructing a field association map of association business field semantics, rule logic and historical operation habits corresponding to the rule matrix; and when new service information is accessed, generating processing prompt information according to the field association map and the service information. According to the method, the intelligent level of a complex business process is fundamentally improved, and information processing pain points in the fields of finance, medical health, old-age care and the like are effectively handled.

This invention discloses a Bayesian inversion extraction method for the equilibrium signal of wide-span altimeter data. The method involves acquiring and preprocessing sea surface height anomaly sequences from radar interferometers and nadir altimeters. A normalized sine square window function is applied for windowing, and multidimensional spatial averaging is used to estimate the one-dimensional wavenumber power spectrum. A piecewise power law-based equilibrium signal spectrum model and a noise spectrum model constrained by dynamic sea state are constructed, and the set of spectral parameters is extracted through logarithmic domain weighted least squares fitting. A set of spatial covariance matrices is constructed using cosine integral transform and Abelian forward and inverse transforms. A graphics processor is scheduled to perform batch matrix decomposition and singular fault-tolerant regularized inversion to solve for the posterior mean vector and posterior covariance matrix of the target equilibrium signal. Window fusion and index mapping are applied to fill the gaps in nadir observations. Geostrophic dynamics parameters are calculated, uncertainty quantification is performed based on the linear error propagation law, and the knowledge base is updated based on the exponential moving average algorithm. This invention achieves suppression of observation noise and physical filling of observation gaps, improving the adaptability of the inversion system to environmental changes while preserving non-Gaussian dynamic characteristics.

The invention discloses a power transmission line fault identification method and system based on an improved PSO-SVM incremental learning strategy. The method comprises the following steps: training a PSO-SVM model by using historical fault data of a power transmission line to determine a support vector; calculating the importance degree of the support vector and the sparsity of the K-neighbor center to screen out playback data; merging the newly added fault data and the playback data into an incremental data set; new samples are generated in a sparse sample distribution area in the incremental data set through a zoning oversampling technology to achieve intra-class data balance of the incremental data set, then the incremental data set is used for training the PSO-SVM model again through multi-task cross integration learning to achieve inter-class data balance of the incremental data set, and a final fault recognition model is obtained. According to the method, the self-adaption and high-precision identification of the fault is realized under limited computing resources.

This invention belongs to the field of port operation and management technology. It proposes a ship behavior recognition method and system based on the LightGBM multi-classification algorithm. The method involves preprocessing AIS refueling behavior data and limited navigation mark data, performing feature extraction, feature selection, data standardization, and data balancing. This yields specific ship refueling behavior labels corresponding to the AIS refueling behavior data, enabling the model to effectively identify all refueling behaviors of various types of ships during operation and improving the accuracy of ship refueling behavior recognition. The LightGBM algorithm, through an efficient gradient boosting decision tree framework combined with feature extraction, feature selection, and data balancing, can accurately distinguish between different refueling and non-refueling behaviors of ships, improving the recognition accuracy of all ship refueling behaviors.

The invention relates to the technical field of electric energy metering anomaly detection and diagnostic analysis, in particular to a distributed dynamic electric energy metering-oriented unbalanced data processing method and system, and the method comprises the steps: carrying out the generation of unbalanced data through employing an anomaly generation model for the abnormal data in multi-dimensional electric energy data; the anomaly generation model is used for adopting a voltage or current phase as an input item, carrying out data balance aiming at the acquired unbalanced type data, and realizing simulation amplification of small sample anomaly data by utilizing a self-adaptive synthesis model aiming at small type sample data; according to the method, the adaptive synthesis data sample and the data conversion strategy under the drive of data exception model analysis are introduced, so that the detection capability of minority class exception modes is remarkably improved, the limitation of a traditional model on an unbalanced data set is overcome, and the problem of data imbalance is effectively solved.

The application provides a kind of municipal maintenance work order overtime degree prediction method and system, the method first obtains municipal maintenance work order data and divides discrete feature data therein into fixed class type data and fixed sequence type data, then label, one-hot encoding and leave-one-out method encoding are used to encode fixed class type data and fixed sequence type data respectively, then the continuous feature data in municipal maintenance work order data is normalized, and the normalized and encoded data is used as training set sample, then SMOTE+Tomeklinks method is used for data balancing, the balanced training set sample is trained to establish gradient boosting tree model, and the gradient boosting tree model is adjusted using bayesian optimization algorithm, then the learning curve is used to test the generalization performance of the gradient boosting tree model, the gradient boosting tree model that passes the generalization performance test is used as the optimal gradient boosting tree model, and then the overtime degree of municipal maintenance work order is predicted, which effectively improves the efficiency of work order processing.

The application discloses a thrombus feature analysis method and system based on deep learning of lower limb vein data, solves the problems of insufficient lower limb thrombus feature recognition and attention focusing deviation. The method comprises the following steps: acquiring an ex vivo lower limb vein data set containing original data and a thrombus region segmentation mask; preprocessing and extracting key layer data covering the thrombus region as analysis data, and constructing a double-channel tensor of the data and the mask; balancing the data and dividing the data into training, verification and test sets; inputting an improved EfficientNet-B3 (additional fine feature convolution layer) and a mask guided multi-head self-attention module to extract thrombus features and focus on the thrombus region, and outputting a classification identifier representing thrombus attributes; generating a thrombus region visual attention distribution map based on the identifier, and synchronously outputting thrombus bounding box, area and other parameters. The system comprises preprocessing, data balancing, model training and reasoning, and result output modules. The application improves the thrombus feature analysis accuracy and result interpretability, and is suitable for automatic analysis of lower limb vein thrombus features.

The invention provides a knowledge-driven copper concentrate grade grading prediction method, and relates to the technical field of mineral dressing and smelting and copper concentrate grade grading prediction. Aiming at the problems of characteristic and label time granularity difference, high-grade sample scarcity, characteristic high dimension and sensitivity difference and category imbalance in copper concentrate grade grading prediction, pseudo labels are generated based on a variational auto-encoder VAE, dynamic threshold screening is carried out on the pseudo labels, data balance processing is carried out on a screened data set based on an FW-SMOTE algorithm, and grade grading prediction of the copper concentrate is realized. Obtaining a balanced data set; constructing a stack integration model based on sensitivity identification, wherein the stack integration model comprises a plurality of base models, a high-sensitivity feature correlation sample screening module and a meta-learner; a balance data set is used for carrying out multi-round cross training on a stack integration model based on sensitivity identification, the generalization ability of the model is optimized, overfitting is avoided, meanwhile, a grading result is output according to the national standard, different smelting processes are directly adapted, and innovativeness and industrial practicability are both considered.

The present application relates to a kind of VAE-SAGAN model, fraud detection model training method and system.VAE-SAGAN model training method includes: obtaining multiple first fraud samples as first training set;The first training set is input into pre-training VAE-SAGAN model for training, and training process is as follows: each first fraud sample is input into the pre-training VAE-SAGAN model, and each first reconstruction fraud sample is obtained;Obtain the reconstruction loss value and the countermeasure loss value between each first fraud sample and corresponding first reconstruction fraud sample;The sum of the reconstruction loss value and the countermeasure loss value corresponding to each first fraud sample is used as total loss value;Based on each total loss value, the parameter of the pre-training VAE-SAGAN model is adjusted, and the training is repeatedly executed until each total loss value converges, and / or, until iteration is preset number, then the current pre-training VAE-SAGAN model is used as target VAE-SAGAN model.The effect of data balancing between fraud sample and normal transaction sample can be ensured by oversampling method.

The application discloses an abnormal user detection method in an online dating APP, first, a trust model of a user is comprehensively constructed; the result output by the trust model is processed again through data balancing; then, normal users and malicious users are labeled, and a data set suitable for a graph neural network is constructed; the constructed data set is trained by using a GraphSage method in the graph neural network, a random sampling of neighbors is adopted to control the size of a K-order subgraph of a node, and on this basis, the sampled subgraphs are randomly combined to complete the training; finally, the trained graph neural network model is used for detecting new user data. The method fully combines personal information, conversation information and interaction information of a user to construct a trust model, and by using the powerful recognition function of the graph neural network, the malicious user in the online dating APP can be automatically distinguished. The recognition accuracy is obviously improved.

The invention discloses a multi-battery data balancing system and method. The multi-battery data balancing method executed by the multi-battery data balancing system comprises the following steps: each of a plurality of battery modules transmits own battery data to other battery modules in a wired or wireless manner; and comparing the plurality of battery data of the plurality of battery modules by each battery module to determine whether to modulate the output voltage of the battery module, if so, modulating the output voltage, and if not, not modulating the output voltage.

The application provides a multi-center data standardization method for evaluating AI ultrasonic model performance, relates to the technical field of ultrasonic data processing, and comprises the following steps: collecting original ultrasonic image data from different data sources, and performing standardization preprocessing to obtain standardization preprocessing images; a multi-task quality evaluation model is constructed and pre-trained; the pre-trained multi-task quality evaluation model is applied to quantitatively evaluate the images to obtain quality evaluation indexes; based on the quality evaluation indexes, a batch-by-batch adaptive threshold strategy and a multi-level screening mechanism are adopted to perform quality screening to obtain qualified labeling images; and the qualified labeling images are sampled in layers to obtain a standardized multi-center ultrasonic evaluation dataset. The application can solve the problems of quality evaluation difficulty caused by heterogeneity of multi-center ultrasonic data sources, inability of a fixed threshold strategy to adapt to quality baseline differences of different data sources, lack of spatial position consideration in artifact evaluation, and absence of a multi-center data balanced sampling mechanism.

This invention provides a method and apparatus for image recognition of small components of transmission line towers. The method includes acquiring pre-annotated image information of transmission line tower components and constructing an image dataset; performing data augmentation and data balancing on the image dataset to obtain a preprocessed image dataset; performing clustering operations on the preprocessed image dataset to obtain predicted anchor frame sizes; constructing a convolutional neural network and training the convolutional neural network using the preprocessed image dataset to obtain a network recognition model; identifying components in the currently acquired transmission line tower images and transmitting image data of identified defective components back to the operation and maintenance center. This invention addresses the problems of existing technologies, such as wasted human resources, low efficiency of manual inspections, difficulty in deploying algorithm models, inability to meet the needs of the power industry, and various other issues with existing image recognition algorithms.

State of art techniques hardly provide data balancing for multi-label multi-class data. Embodiments of the present disclosure provide a method and system for identifying cardiac abnormality in multi-lead ECGs using a Hybrid Neural Network (HNN) with fulcrum based data re-balancing for data comprising multiclass-multilabel cardiac abnormalities. The fulcrum based dataset re-balancing disclosed enables maintaining natural balance of the data, control the re-sample volume, and still support the lowly represented classes there by aiding proper training of the DL architecture. The HNN disclosed by the method utilizes a hybrid approach of pure CNN, a tuned-down version of ResNet, and a set of handcrafted features from a raw ECG signal that are concatenated prior to predicting the multiclass output for the ECG signal. The number of features is flexible and enables adding additional domain-specific features as needed.