146 results about "Data balancing" patented technology

A data storage device comprises at least one array of memory elements arranged in a plurality of rows and columns; coding means for coding an input data into a form having a balanced proportion of "1's and "0's, said coding means comprising means for applying an output of a pseudo random bit sequence generator to said incoming data, wherein the coded data is stored in the array of memory elements such that the "1's and "0's are spatially distributed relatively evenly across the plurality of memory elements; and decoding means for decoding the coded data read from the plurality of memory elements, into the original data.

The present invention relates to an image automatic marking method based on Monte Carlo data balance. The method comprises the steps of carrying out the region segmentation on the training sample images in a public image library, enabling the segmented regions possessing different characteristic description to correspond to one marking word, then carrying out the Monte Carlo data balance on the different types of image sets, extracting the multiscale characteristics of the balanced images, and finally inputting the extracted characteristic vectors in a robustness least squares increment limit learning machine to carry out the classification training to obtain a classification model in the image automatic marking; for the to-be-marked images, carrying out the region segmentation on the to-be-marked images, adopting the same multiscale characteristic fusion extraction method and inputting the extracted characteristic vectors in the least squares increment limit learning machine to obtain a final image marking result. Compared with a conventional image automatic marking method, the method of the present invention enables the images to be marked more effectively, is strong in timeliness, can be used for the automatic marking of the large-scale images, and possesses the actual application meaning.

The invention provides a balance method of actual scene linguistic data and finite state network (FSN) linguistic data. In order to train the language model of a continuous speech recognizer, training linguistic data is produced according to the application field of the speech recognizer. Linguistic data mainly has two resources: one part is an actual scene linguistic data which is obtained by tidying records in actual scene, and the other part is FSN linguistic data generated with a finite state network syntactic rule method. Two linguistic data balance methods are mainly researched in the invention, and the invention provides a method that probability comparison of the keyword shared by the actual scene linguistic data and the FSN linguistic data is taken as basis, and a certain multiple of parts of actual scene linguistic data are used for expanding the FSN linguistic data to obtain the final method of language model training linguistic data. The language model for training linguistic data, which is obtained with the method, greatly improves the recognition performance of the continuous speech recognizer.

The invention discloses a text topic classification model based on multi-source-domain integrated migration learning. The model is composed of a target domain data module, a tagging module, an integrated learning module for multi-source-domain tag determination and a correct data module. According to a classification method for the text topic classification model based on multi-source-domain integrated migration learning, first, data without class tags is classified through the tagging module; and next, data with tags is determined, the data correctly classified through three classifiers is selected and added into the target domain data module, classification is performed through the three classifiers to obtain data with dummy tags and different types of text topics, one type of text topics is selected to serve as target domain data, other types of text topics are used as source domain data and added into the target domain data, and a Softmax classifier is used to test the correct rate. In this way, the negative migration phenomenon brought by single-source-domain migration is effectively avoided, data composition comes from all aspects of a target domain, and data balance can be better met.

The invention discloses a student poverty state prediction method based on data excavation. The method comprises: obtaining learning data of all students at school, consumption data information, and proportion of poor students and non-poor students; dividing student data information into a training set and a test set, and proportion of poor students and non-poor students in each set is consistent with a determined proportion; using an oversampling algorithm to perform data balance on the data in the training set; using a random forest algorithm to perform model training on the training set, testing and evaluating the model by the test set, to obtain a student poor state prediction model in optimal performance; and using the student poor state prediction model to predict the poor state of the students. The method performs comprehensive examination and prediction on poor state of the students through objective data and behaviors of the students at school, so as to prevent subjectivity and randomness in student poor state evaluation. The method is scientific and practical, and can rapidly perform algorithm solution and data analysis.

The invention discloses a P2P borrower credit evaluation method based on big data. The invention comprises a data acquisition module, a data processing module and a model building module. In the era of big data, credit data sources are expanding, mainly including the following four aspects: credit data generated by financial institutions, credit data generated by relevant government departments, credit data generated by other public utilities, Internet credit data generated by the network. The data module is mainly divided into two parts, the credit data generated by financial institutions, relevant government departments and public utilities are qualitatively defined as structured data collection; Social media data, such as WeChat friends and Sina Weibo, are collected as unstructured datain Internet credit data. Data processing module is mainly aimed at structured data, including data balance processing and feature selection. As that imbalance phenomenon exist in the structured dataof personal credit, the invention uses CART-SMOTE algorithm for data balance processing; Under the background of big data, the characteristics of personal credit evaluation data are complicated, and irrelevant and redundant variables will have adverse influence on the accuracy of model prediction. The invention uses random forest and gradient descent decision tree to select evaluation characteristics. The structured data model uses an improved lightGBM for preliminary credit ratings; Feature extraction from unstructured social text data, credit evaluation and affective tendency analysis usingin-depth learning. Then the emotional tendencies in personal social media text data are fed back to the credit evaluation of P2P borrowers to study the correlation between them. Provide a reference for the final credit evaluation structure.

The embodiment of the invention discloses a method for data balancing. The method comprises the steps that a load degree and n weight values of n weight factors of each virtual node on m physical nodes in a distributed database system are acquired, wherein m and n are integers which are larger than 1; according to the load degree and the n weight values of the n weight factors, weight averaging isconducted, so a weight coefficient of each virtual node is obtained; according to the weight coefficients of the virtual nodes correlated with each physical node, a standard zoning number is obtained; whether data distribution satisfies data balancing conditions is judged according to the standard zoning number corresponding to each of the m physical nodes; and if the conditions cannot be satisfied, data balancing processing can be conducted to the m physical nodes. The embodiment of the invention also discloses a data balancing device. By the method and device disclosed by the invention, thedata balancing of the physical nodes in the distributed database system can be conducted, and resource configuration can be optimized.

The invention discloses a data increment method and device, computer equipment and a storage medium, and the method comprises the steps: obtaining a scene classification sample corresponding to a specific scene and a specified sample proportion, carrying out the text preprocessing of the scene classification sample through employing a regular expression, and obtaining a to-be-trained text; carrying out incremental training on the to-be-trained text by adopting the original word vector model to obtain a target word vector model; based on the actual sample number corresponding to each classification label and the total sample number corresponding to the scene classification samples, determining an actual sample proportion corresponding to the classification labels; if the actual sample proportion is smaller than the specified sample proportion, taking the scene classification sample corresponding to the classification label as a sample to be incremented; and inputting the to-be-incremented sample into the target word vector model for processing, obtaining candidate phrases corresponding to the to-be-incremented sample, randomly selecting one target synonym from each candidate phrasefor replacing the to-be-incremented sample, and obtaining a first newly added sample. The method can effectively guarantee data balance.

The invention belongs to the technical field of emotion calculation and emotion recognition, and particularly relates to an electroencephalogram signal emotion recognition method based on a dimension model. Aiming at the technical problem of low positive and negative emotion classification accuracy under a dimension model at present, the electroencephalogram signal emotion recognition method based on the dimension model comprises the following steps: (1) electroencephalogram preprocessing; (2) respectively extracting frequency domain, time frequency and nonlinear characteristics of the preprocessed multi-lead electroencephalogram signals; and (3) conducting emotion classification of a stack type self-encoding neural network. The classification method provided by the invention is stable and reliable, explains the influence of data balance, feature combination and emotion label threshold on electroencephalogram signal emotion recognition, and improves the accuracy of electroencephalogram signal positive and negative emotion classification under the dimension model.

The invention relates to an HBase-based Internet-of-things big data storing and accessing method, and belongs to the technical field of Internet-of-things data processing. The method includes the following steps that 1, HBase lists are created, each terminal type corresponds to one corresponding HBase list, the name of each terminal type serves as the name of the corresponding HBase list, and column family names (Column Family), column qualifier names (Column) and a Region splitting strategy are specified; 2, reported data is introduced to the HBase lists, and a set of rowkey generation scheme is designed; 3, data query is conducted. The method fully considers the features of Internet-of-things application fields and the characteristics of HBase column type storage, efficient storage of the data is achieved, requirements of users under different application scenarios are met, and the scalability and data balancing of a system are effectively taken into account.

The invention discloses a vector contour line data partitioning method with the space proximity relation considered. The method comprises the steps of (1) reading contour line data and conducting quantitative statistics on the characteristics of the contour line data, (2) calculating the coordinates of the central point of the minimum enclosing rectangle of each contour line and expressing vector contour line data with a three-dimensional point provided with elevation information, (3) setting the number K of parallel computational nodes, (4) calculating the load threshold of each computational node in an ideal load balanced state and calculating the lower limit and the upper limit of the load thresholds, (5) selecting M (M=20K) points to serve as initial clustering central points, (6) clustering point features into M class clusters, (7) recalculating the coordinates of the central point of the M class clusters, (8) expressing the M class clusters with tetrads, (9) taking the tetrads as minimum data partitioning units and clustering the M tetrads into K class clusters, and (10) the end. According to the method, the data balancing principle is met, load balancing is guaranteed, and a high spatial clustering degree of partitioned data is guaranteed.

The invention discloses an orthogonal frequency division multiplexing spread spectrum underwater acoustic communication pilot-free decision feedback channel estimation method under the sparse channel condition. The method comprises the steps that 1) OFDM frequency domain information extraction is performed; 2) the extracted frequency domain information in the step 1) is transmitted to a spread spectrum identification module to perform spread spectrum identification; 3) the spread spectrum identification result of the step 2) is transmitted to a decision feedback loop module, iterative learning of an underwater acoustic communication is performed and the channel is reconstructed; and 4) despreading outputting of the data balanced by using the reconstructed channel is performed. The spread spectrum signal is effectively identified by using the sparse characteristic of the underwater acoustic communication channel, and decision feedback estimation is performed on the underwater acoustic communication channel response by using the identification result so that reliable reconstruction of the underwater acoustic communication channel can be realized, balancing of the underwater acoustic communication signal can be reliably realized without loss of the communication quality and the underwater acoustic communication rate can be effectively enhanced.

The invention discloses a data balancing method of a storage system. The method comprises the following steps: counting the number of object storage devices in the storage system and the number of minimum storage units in each object storage device; calculating a total sum of the numbers of the minimum storage units in the object storage devices; calculating a distribution average value of the minimum storage units; judging whether the minimum storage units are uniformly distributed according to a preset balancing rule, if not, judging whether all preset weight adjustments are accomplished, if not, performing weight adjustment on the object storage devices according to a preset adjustment rule, and distributing the minimum storage units again after the adjustment; and counting the number of the minimum storage units in each object storage device again, and returning to a balancing judgment operation. The invention further provides a data balancing apparatus of the storage system. According to the data balancing method, the defect of unbalanced data distribution is improved by adjusting the weights of the OSDs so as to guarantee that capacities of the OSDs in storage pools are basically the same and improve the whole capacity utilization rate of the system.

The invention discloses a Wifi probe and a data balanced collection system and method. The data balanced collection method includes the steps that the Wifi probe sequentially scans Wifi channels in a detection area of the probe in a rotary mode to obtain broadcast data packages of devices in the Wifi channels; the Wifi probe uploads the obtained broadcast data packages to a server. By means of the rotary scanning mode of the Wifi channels, different scanning time is allocated to different Wifi channels according to a single factor or multiple factors so that the balance degree of data collection can be guaranteed to the largest extent; before the data is uploaded to the server, the data is preprocessed, invalid information in the data is removed, only valid information is kept, the data transmission magnitude can be greatly reduced, the transmission cost is reduced, and subsequent quick and accurate data analysis is benefited.

The invention provides a data balancing method and device and a management device in a distributed storage system. The method comprises the steps of determining a target storage node from a pluralityof storage nodes when a data write request from a client is received, thereby enabling the client to write target data corresponding to the data write request into the target storage node; determiningat least one backup storage node from the rest storage nodes different from the target storage node in the plurality of storage nodes based on storage data quantity in each storage node; and sendingidentification information of the at least one backup storage node to the target storage node, thereby enabling the target storage node to write the target data into the at least one backup storage node. According to the data balancing method and device and the management device in the distributed storage system provided by the invention, the data balance problem is taken into consideration when the data is written, so data imbalance situations are greatly reduced.

The invention provides a data balancing strategy and multi-feature fusion-based image labeling method. The method comprises the following steps of: 1, carrying out semantic grouping on training imagesto obtain semantic groups; 2, extending the semantic groups by adoption of a data balancing strategy; 3, inputting the training images into a trained deep convolutional neural network to obtain a deep feature of each image in each semantic group; 4, calculating multi-scale fusion feature of each image in each semantic group; 5, carrying out multi-feature fusion on the multi-scale fusion feature and the deep feature so as to obtain a fused feature of each image in each semantic group; 6, extracting a shallow feature and a deep feature of a to-be-tested image and carrying out feature fusion toobtain a fused feature of the to-be-tested image; and 7, calculating a visual similarity between the fused feature of the to-be-tested image and the fused feature of each image in each semantic group,and sorting the visual similarities to obtain an image labeling result and then obtain a category label. According to the method, the problems that training set images are unbalanced and the single feature expression ability is not strong are solved.

The invention discloses an unbalanced data sampling method and device under federated learning, and the device comprises a data monitoring module, a data balancing module, a local training module, anda central server. An unbalanced data balancing scheme based on mixed sampling comprises the steps: according to the unbalance proportions of different data sets, obtaining a balance data set based ona mixed sampling method combining a few synthesized samples and a clustering down-sampling integration method; in combination with data set real-time monitoring, automatically setting an unbalanced data set processed and timely updated in a federated learning scene; therefore, through an unbalanced data set processing method combining a data level and an integration mode, the capacity of the dataset is fully utilized, and automatic equalization and updating of the unbalanced data set are realized by detecting the data set through data change detection.

The invention is suitable for the fields of biomedical signal processing, and provides an arrhythmia detecting method, device and terminal. A training set is reconstructed in a heart beat waveform list of an intercepted electrocardiosignal through the arrhythmia detecting device, feature studying and sorting are conducted on heart beat waveform and an RR interval in the training set based on a deep neural network after the number of the heart beat waveform list in the training set is amplified to determine the type of arrhythmia; the improvement of the expansion of training samples and data balance is achieved by the reconstruction and amplification of the training set to make feature study and sorting of heart beat signals performed by the deep neural network convenient, the automatic detection of arrhythmia is achieved, and the detection efficiency of the arrhythmia is improved, man-made interference is reduced, and the accuracy of arrhythmia detection is improved.

The invention discloses a QoS (Quality of Service) control method based on Ceph distributed storage Osd end data recovery. A QoS mechanism is mainly added to a distributed storage Osd server; controlling is carried out according to a byte stream method; a token bucket is arranged at each Osd end, wherein the maximum number of the tokens is the maximum current limiting size of the Osd server, whether the number of the tokens is enough or not is judged when the data is balanced each time, if so, a request IO is started to be issued, and if not, a background process is waited for to issue the tokens to a token bucket, and when the tokens are sufficient, a waiting IO request is started to be issued. By controlling the token issuing rate of the background process, the purpose of accurately controlling the data flow can be achieved, and meanwhile, the QoS has the effect of coping with the burst IO, so that the data balancing process is closer to the scene of controlling the data flow by a user scene.

The invention discloses a positive and negative sample data balancing method in factory PCB defect detection, which is a data balancing method in PCB positive and negative sample classification basedon an adversarial generative network, and mainly comprises the following steps: collecting, sorting and classifying a data set; designing an encoder which is composed of five convolution layers, and extracting features from the input image by the encoder; designing a converter which is composed of eight residual blocks and converts the feature vector from a source domain to a target domain; designing a decoder, wherein the decoder is composed of five deconvolution layers; designing a discriminator, wherein the discriminator is composed of seven convolution layers; designing a loss function, wherein the loss function comprises four parts; preparing a training set for model training; the obtained weight file is used for a test set, and a negative sample needing to be amplified is synthesized. The method is high in robustness, wide in application range and excellent in synthesis effect. And by means of cyclic consistency conditions, the effect of standardizing the model is achieved, and the generation effect of the shape and texture of the synthesized image is flexibly controlled to a certain extent.

The invention discloses a continuous time balance circuit applied to a high-speed serial interface. The continuous time balance circuit comprises a programmable matching resistor module which is coupled to the ground, a continuous time balance amplifier circuit and an imbalance calibration module, wherein an external data signal is connected with the programmable matching resistor module through direct-current coupling or alternating-current coupling to generate locally received signals INN and INP; the signals are subjected to data balance through the continuous time balance amplifier, and meanwhile, direct-current level conversion is finished; the unbalanced data signals INN and INP which are referenced to the ground are converted into balance data signals OUTN and OUTP which are referenced to a power supply; and meanwhile, the system imbalance is measured by the imbalance calibration module; and the output Ioffsetn and the output Ioffsetp of the imbalance calibration module are regulated, so that the imbalance and removal are finished. According to the continuous time balance circuit, the three functions of level conversion, imbalance calibration and balance amplification of the data are realized by utilizing the continuous time balance amplifier at the same time; the error code rate of the data transmission is reduced; and the power consumption and the area of an integrated circuit are reduced.

An embodiment of the invention discloses a load balancing method for processing MapReduce data skew, and relates to the field of cluster dispatching and load balancing. As large-scale MapReduce clusters are widely used for processing big data, one of current main problems is how to furthest shorten working time and improve MapReduce service efficiency, and data balancing related problems are less related in the past research of MapReduce, so that a load balancing algorithm of a Reduce end is provided to solve the problem of data skew in the running process of the MapReduce. The method includes the steps: performing sampling analysis for inputted data, and determining the average task number of each Reduce node; performing descending sort from big to small according to the task number based on a time coefficient, and performing sort according to a sequence number if the numbers are the same; sequentially distributing tasks according to the principle of maximum resource surplus capacity and a sorted task sequence until all tasks are completely distributed; submitting a distributing mode to a self-defined Partition function, and executing a processing process.

The invention discloses a design method for realizing an Hbase database remote sensing big data storage model based on Google S2. The method comprisesa; achieving grid cutting of remote sensing data through a Google S2 algorithm, partitioning, fragmentation and ground space discretization are conducted on a large-range whole remote sensing image, and storage management of image data is facilitated; through establishment of a table storage model of the Hbase database, attribute expression of the partition remote sensing image in multiple dimensions is achieved, and data structure integration ofmulti-source heterogeneous remote sensing image data is achieved in a data band discrete storage mode. Characteristics of different types of remote sensing image data are fully considered, efficientstorage of remote sensing big data is achieved, requirements of users under different application scenarios are met, and expandability and data balance of the system are effectively considered.

The invention discloses a diabetes retinopathy image multi-classification method based on deep learning, and the method comprises the following steps: firstly, obtaining an original data set, and carrying out the preprocessing of the original data set; then carrying out category data balance on the preprocessed data set; then, carrying out migration training on the data set with balanced categorydata to obtain a diabetic retinopathy image multi-classification model; and finally, inputting the sample to be detected into the diabetic retinopathy image multi-classification model for prediction,and completing classification of the diabetic retinopathy image. The method provided by the invention has the advantage of being convenient to use, and in addition, the classification accuracy is improved by carrying out migration training on the data set after the class data is balanced.