171 results about "Open source data" patented technology

The invention discloses a knowledge graph analysis and question answering method and device based on a large language model. The method comprises the steps of obtaining original data, performing data processing based on a large language model to obtain target data, constructing a knowledge graph according to the target data, and storing the knowledge graph into a graph database; acquiring an open source data set, and processing the open source data set by using the large language model to obtain a training data set; according to the graph database, constructing a dictionary which is used for reflecting a graph structure of the graph database; utilizing the training data set, the dictionary and a preset prompt instruction to finely adjust the large language model to obtain an optimized large model; generating a query statement by utilizing the graph database and the optimization large model according to a query problem of a user, and determining a query result in the graph database according to the query statement; and based on the optimization large model, carrying out integration processing on the query question and the query result to obtain a final answer. According to the method, the construction process of the knowledge graph is simplified, and the accuracy of analysis and question answering is improved.

A cooling system running method includes obtaining an open source data set of a cooling system including components each being configured with parameter(s). The open source data set includes a parameter value corresponding to each parameter, and the parameter(s) configured for one component include a controllable running parameter. The method further includes determining an initial model for predicting power consumption of the cooling system, selecting a target algorithm using an open toolbox, training the initial model using the open source data set and the target algorithm to obtain a target model, deploying the target model to a controller of the cooling system, and causing the controller to at least control operation of the one component according to a parameter value corresponding to the controllable running parameter that is defined in a power consumption optimization policy determined by the controller using the target model.

The invention discloses a strategy question-answering system and method based on a vertical domain large model, and the method comprises the steps: S1, receiving the natural language input of a user, and retrieving an open source data set to obtain an original data set with the comprehensive similarity meeting the requirements; s2, performing necessary data cleaning and preprocessing on the original data set to obtain a training set; s3, performing fine tuning on the pre-processed basic large language model by adopting the training data set, and introducing a low-rank structure to modify a weight matrix to obtain a final model; s4, the problem enters a final model for post-processing to generate prediction output; the system takes a large language basic model subjected to fine tuning training as a core, introduces a low-rank structure to perform fine tuning on model parameters, reduces parameter quantity required by training while keeping model performance, reduces model complexity, reduces an overfitting risk, improves generalization ability, supports deep fusion of a specific field knowledge base and universal field data, and has a wide application prospect. And the accuracy and correlation of questions and answers are improved.

The invention discloses an intelligent content generation system based on open source data analysis, and relates to the technical field of monitoring analysis, and the system comprises a data preprocessing module which is used for carrying out the multi-modal content collection of a heterogeneous open source data platform, and obtaining a structured semantic feature matrix; the graph construction and extraction module is used for constructing a domain knowledge graph based on the structured semantic feature matrix to obtain a domain evolution feature vector; the intelligent content generation module is used for inputting the domain evolution feature vector into a multi-modal generative adversarial network for semantic-visual cross-modal mapping to obtain style-enhanced intelligent generation content; and the optimization module is used for performing propagation effect simulation prediction on the style-enhanced intelligent generation content, generating a content propagation influence matrix, performing optimization iteration on the intelligent generation content according to the content propagation influence matrix, generating final optimization content and pushing the final optimization content to a target terminal.

The invention relates to the technical field of risk management, in particular to a multi-scale urban rainstorm flood risk assessment method. The method comprises the following steps: acquiring a regional basic data set; performing data preprocessing on the regional basic data set to generate a regional basic preprocessed data set; constructing a rainstorm flood risk assessment index system based on the regional basic preprocessing data set to obtain an urban scale flood assessment index, and performing urban scale rainstorm flood risk assessment to obtain an urban scale rainstorm flood risk assessment result; according to the method, the index information is obtained from the open source data, and the defects that in previous researches, part of index data is poor in spatial precision and long in time are overcome; the multi-scale urban rainstorm flood risk assessment method comprehensively considering the urban scale and the block scale is provided, and the accuracy, scientificity and comprehensiveness of urban rainstorm flood risk assessment can be improved.

The present application provides a translation model training method, translation method, device, electronic equipment, and medium, relating to the field of translation model technology. The translation model training method includes: expanding vocabulary of the translation model with commonly used vocabulary and specialized vocabulary in advance, optimizing feature vectors in the vocabulary based on a large language model, cleaning an open-source dataset using a data cleaning method to filter and retain corpora meeting a quality standard, and training the translation model using filtered corpora. The translation method includes: obtaining source language text to be translated, inputting the source language text into a translation model to obtain target language text after translation. The embodiments of this application not only enhance the vocabulary of the translation model but also improve the accuracy of the translation model in specialized domains.

The invention discloses a recommendation system deviation correction method fusing big language model world knowledge, which comprises the following steps: acquiring an open source data set of a recommendation system, and generating a training set; guiding a large language model to generate description information of an article and preference reasoning of a user through a training set in combination with a context learning technology; encoding the description information of the article and the preference reasoning of the user through a pre-trained text encoder to generate a first encoding vector, and encoding the IDs of the user and the article to generate a second encoding vector; inputting the first coding vector and the second coding vector into a multi-mode expert network module to obtain final representations of the user and the article; inputting the final representations of the user and the article into a prediction layer to obtain a prediction value, and optimizing a multi-modal expert network module through a cross entropy loss function; and performing recommendation system correction through the optimized multi-mode expert network module. The inherent deviation in the user behavior data is eliminated, the real preference of the user is captured, and recommendation system deviation correction is completed.

The invention discloses a hybrid photovoltaic prediction method based on SARIMA-LSTM, and the method comprises the steps: carrying out the calculation according to historical photovoltaic power generation data in an SCADA system, wherein the historical photovoltaic power generation data comprise the power generation power and the accumulated power generation amount, and the meteorological data comprise the irradiance, the temperature, the humidity, the wind speed, the cloud cover and the like, and are obtained from a meteorological station or an open-source data platform; statistics is carried out by combining an SARIMA model with a seasonal prediction function and an LSTM model for processing a time sequence, and a more accurate photovoltaic power generation prediction value is obtained after combined prediction and evaluation. Compared with the prior art, short-term prediction and long-term prediction can be considered, and seasonal change data prediction is more accurate.

The invention belongs to the technical field of extreme ocean phenomenon forecasting, and discloses a typhoon and storm surge lightweight rapid forecasting system and method. The method comprises the following steps: preprocessing a multi-source marine environment data set and a typhoon moving path; constructing a typhoon and storm surge data set based on the preprocessed data; based on the constructed typhoon and storm surge data set, establishing a deep learning model VIT-TSS based on VIT; and comparing the output result of the deep learning model VIT-TSS with the open source data and the measured data to verify the forecast precision. The typhoon and storm surge intelligent forecasting model provided by the invention can realize typhoon path forecasting and regional water increase forecasting. According to the VIT-TSS model provided by the invention, the prediction time of the model is greatly shortened while the calculation precision is improved. The system provides key information for emergency management and navigation.

The invention relates to the technical field of power grids, in particular to a vertical domain distribution network planning method and system based on a large language model. The method comprises the following steps: collecting and arranging a distribution network structure and operation data, and constructing a'graph 'management system; extracting historical planning schemes to form a typical case library; fusing external open source data and source side data to establish a big database and an access interface; constructing a bilateral prediction model and a planning scheme generation model based on machine learning; carrying out secondary pre-training and practical packaging on the open source large language model; planning parameters are obtained in a guiding mode in a question-answer mode, and a scheme is generated. According to the method, the problems of high labor cost, long planning time, large error, unstable planning quality, poor interactivity, high learning threshold and the like of traditional distribution network planning are solved, the planning efficiency and precision are improved through an intelligent technology, the professional threshold is reduced, a brand new solution is provided for coping with the new situation of high permeability of new energy and electric energy replacement acceleration, and the method is worthy of popularization and application. And the practical value is remarkable.

The invention relates to the technical field of big data, and discloses a data processing method and device, computer equipment, a storage medium and a program product. The method is applied to a NiFi cluster, and comprises the following steps: acquiring first target data; wherein the first target data is used for indicating changed data; performing type conversion processing on the first target data to obtain second target data; performing field mapping on the second target data to obtain third target data; and writing the third target data into an open-source data lake storage framework in a java client mode.

The invention relates to a power battery and energy storage battery health state prediction method and system, and the method comprises the following steps: S1, obtaining original EIS data through an open source data set, extracting the center position, radius and arc length of a medium-high frequency Nyquist diagram arc, and extracting the length of a low-frequency Nyquist diagram as health features, and obtaining a training set; step S2, constructing a DGPR model; inputting the training set into the DGPR for training, selecting a kernel function, training to obtain a hyper-parameter, and establishing an accurate battery health state prediction model; and S3, inputting the health features extracted in the test set into the trained health state estimation model, and performing battery health state prediction according to hyper-parameters obtained by training. According to the method, the health features can be extracted by intercepting the EIS frequency band data, the health features are input into the DGPR for training to obtain the hyper-parameters, and the health state of the battery is predicted according to the hyper-parameters.

The invention discloses a field data collection method and device, electronic equipment and a storage medium, and relates to the technical field of data processing, and the method comprises the steps: obtaining an open source data set in an open source corpus, and carrying out the field correlation labeling of all data in the open source data set, and obtaining a labeled data set; screening the annotated data set according to the annotated domain correlation to obtain an annotated data subset related to the target domain; performing compliance annotation on each piece of data in the annotation data subset to obtain an annotation data subset; and screening the annotation data subset according to the compliance of the annotation to obtain target domain data. According to the method, professional corpus data which not only has financial field correlation but also meets compliance requirements is efficiently extracted from a large amount of open source corpora, the quantity and quality of data in the target field are ensured, and a reliable data basis is provided for pre-training of a large model in the financial field; therefore, the understanding ability and expression accuracy of the model in the financial field are improved.

The invention discloses a satellite network anomaly detection method based on federated learning, relates to the technical field of satellite network communication, and solves the problems that an existing detection system cannot effectively learn a feature model between data and cannot realize balance of optimal model training and communication overhead. According to the method, an open source data set is preprocessed, a CvT-LSTM model is constructed, a general framework with a single server and K participants is designed, an aggregation strategy based on local data volume and anomaly detection model precision weighting is used, anomaly detection model parameters of a selected client are uploaded to a global model, and after being aggregated by the global model, the anomaly detection model parameters of the selected client are obtained. And issuing to all clients, and the like. According to the method, interference of low-quality data is avoided, transmission and calculation of invalid data are reduced from the source, the communication cost is remarkably reduced, only the high-representativeness client participates in global model aggregation, and the communication burden is reduced to the maximum extent while the model performance is ensured.

The invention discloses an anti-cancer drug sensitivity prediction method and system based on bipartite graph and hypergraph learning, and belongs to the technical field of computer software. The method comprises the following steps: acquiring cell line and drug data and drug and cell line reaction original IC50 values by using an open source database, and standardizing the IC50 values; constructing a bipartite graph of the cell line and the drug, and obtaining embedded representation of the drug and the cell line by applying a graph convolutional network; super nodes are introduced to construct a cell line-drug isomerism hypergraph, and node features are learned through GCN; and finally, fusing the output of the GCN layer by applying an attention mechanism, and predicting the final association probability of the cell line-drug pair in combination with a full-connection layer. According to the method, combined exploration and collaborative optimization of the bipartite graph and the hypergraph are achieved, meanwhile, potential and complex relations between the medicine and the cell line are explored through the hypergraph, the quality of node feature representation is jointly improved, and therefore the association score between the medicine and the cell line is predicted more accurately.

The invention discloses a clothing attribute identification method and system, computer equipment and a readable storage medium, and relates to the technical field of image identification. The clothing attribute identification method comprises the following steps: acquiring a to-be-identified picture; and obtaining a picture text database of the clothes, and obtaining a clothes attribute recognition model according to the picture text database of the clothes and the deep learning model. And according to the clothing attribute identification model, identifying the attribute of the clothing in the to-be-identified picture, and obtaining the attribute data of the clothing in the to-be-identified picture. In the aspect of data, massive clothing data is crawled through a crawler technology, part of open source data sets are combined, pre-labeling is performed by means of the powerful content understanding ability of a visual language large model, and a professional clothing attribute data set is constructed. In the algorithm aspect, the advantages of a Transform architecture and a CNN are combined, a lightweight network structure with a self-attention mechanism based on a local sliding frame and key point regression as an auxiliary task is designed, and the method is used for solving the problem of clothing multi-attribute recognition and giving consideration to model precision and reasoning time delay.

The invention discloses a small target detection method under an unmanned aerial vehicle visual angle based on an SCGD-YOLO network, and the method comprises the following steps: obtaining an open source data set of a small target under the unmanned aerial vehicle visual angle, dividing the data set into a training set, a verification set and a test set, the data set comprising ten types, and the main types being small targets; configuring a network environment required by the network model; constructing an SCGD-YOLO network model, wherein the SCGD-YOLO network model comprises a backbone network, a neck network and a detection head; sending pictures and labels of the training data set into the constructed SCGD-YOLO network model for training, and adjusting corresponding hyper-parameters according to a result of the verification set to obtain an optimal training result; finally, to-be-detected pictures in the test set are sent into the trained SCGD-YOLO network model for small target detection, and a detection result is output, the problems that the small target resolution is low and the detection precision is reduced when the unmanned aerial vehicle carries out high-altitude shooting are solved, the detection precision of the small target is improved, and the parameter quantity of the model is reduced.

The invention relates to a security alarm noise reduction method based on rules and a large model, which belongs to the field of network security and comprises the following steps of: collecting security alarm data and security logs and preprocessing the security alarm data and the security logs; constructing a dynamic comprehensive rule knowledge base comprising a screening rule base and a plug-in knowledge base, and periodically updating the dynamic comprehensive rule knowledge base; selecting an open-source large model of the security field, using the open-source data set of the security field and historical security alarm data, and utilizing a distillation technology to finely adjust a noise reduction small model; a knowledge base driven periodic fine tuning and dynamic knowledge injection double-path coordination mechanism is adopted to improve the capability of the noise reduction small model; and finally, realizing efficient and accurate alarm noise reduction based on the screening rule base and the noise reduction small model. According to the invention, the efficiency and accuracy of security alarm noise reduction are effectively improved, and an efficient, accurate and intelligent alarm noise reduction method is provided for the field of network security.

The invention provides a data processing method and system, electronic equipment, a storage medium and a product, and relates to the technical field of information. The data processing method comprises the following steps: collecting original reconciliation data of each business department, taking a distributed file system as bottom storage, constructing a data warehouse framework and a distributed column-oriented open source database on the basis, and managing the original reconciliation data. Based on a user query instruction, target account checking data of a current checking time period is obtained from an open source database, and the data relates to more than two business departments. And then target reconciliation data are verified, and data of different departments are checked in a cross manner. According to the scheme, a large amount of data is efficiently stored through the distributed file system, ordered management and efficient analysis are achieved through an upper-layer framework and a database, quick and accurate response query can be achieved, the data accuracy and reliability are improved, potential business problems can be perceived in time, and business promotion and financial accounting accuracy are guaranteed.

The invention discloses a method for automatically producing whole-body digital human data, which comprises a first model training stage: collecting a real human speaking data set and acquiring an open source data set as an initial training data set, training a lip shape alignment model and a digital human generation model, and performing data enhancement and data expansion processing based on the initial training data set to obtain a second model training stage; continuing to train the lip shape alignment model and the digital human generation model until the precision of the lip shape alignment model and the digital human generation model meets a preset model parameter threshold value; and a second model training stage: when the data volume of the training data set in the first model training stage reaches a preset data volume threshold value, generating new description text data based on the video data in the training data set, and adding the description text data into the video data set, performing fine tuning on the digital human generation model and replacing the corresponding model in the first model training stage; and executing loop iteration of the first model training stage and the second model training stage at the same time until a preset training target is met.

An instance segmentation model training method includes training the instance segmentation model firstly based on a first data set stored in a first database, and training the firstly-trained instance segmentation model secondly based on a second dataset stored in a second database, where the second dataset includes a large-scale open-source dataset and a segmentation target object-absent image acquired by capturing a working environment within an industrial site.

The invention relates to the technical field of monitoring video processing, in particular to a monitoring video frame identification method based on an image segmentation technology, which comprises the following steps: S1, acquiring a video file: downloading past monitoring videos through a network, and uniformly modifying the monitoring videos into the same format; s2, performing optimization processing on the video: selecting a monitoring video, inputting the monitoring video into optimization software for optimization processing, and improving the definition of an image in the monitoring video; according to the method, a model program can be constructed, the model program performs deep processing on the image through an open source data set in the fields of deep learning and image processing, efficient feature representation can be realized, the accuracy and the fine degree of image semantic segmentation are improved to a great extent, the number of image segmentation is increased, the error segmentation phenomenon is reduced, and the image segmentation efficiency is improved. The phenomena of image edge blurring, wrong segmentation and instable segmentation of a segmentation result generated by a traditional segmentation method are avoided.

The invention relates to the technical field of data processing, and discloses a quotation analysis model construction method and device, equipment and a medium, and the method comprises the steps: obtaining original full-amount data, and labeling the original full-amount data according to a preset structured format to obtain initial full-amount data; wherein the original total data represents complete data including historical quotation data and newly added quotation data on that day; cleaning dirty data based on the initial total data to obtain processed total data; wherein the dirty data cleaning comprises the step of carrying out outlier data screening by utilizing a trained instruction fine tuning model; determining open source data as unstructured data, and comprehensively processing the unstructured data and the processed total data to obtain target training data; and training a large language model by using the target training data to obtain a quotation analysis model. The method has the beneficial effects of standard structured text and real-time response.

The invention discloses a boundary representation generation method based on graph diffusion. The method comprises the following steps: step 1, constructing an industrial part data set and preprocessing the industrial part data set; step 2, constructing and training a Brep-GD model based on the preprocessed self-built data set and the two open source data sets; the Brep-GD model comprises a variational auto-encoder, a graph diffusion model, a continuous topology decoupling model and a post-processing module; 3, reasoning and evaluating the trained Brep-GD model by using a test set of a data set so as to verify multiple types of indexes of distribution measurement, CAD measurement and efficiency measurement of the model; and step 4, applying the evaluated Brep-GD model to generate a simplified B-rep model, the method can significantly reduce redundancy calculation, improve generation efficiency, and ensure that the generated B-rep model has higher geometric accuracy and topology effectiveness.

The invention relates to a traffic facility attribute mining method and system based on multi-modal network open source data. The method comprises the following steps: extracting traffic facility information from a webpage to construct a knowledge graph; extracting positions and appearances of traffic facilities from the images, and analyzing streetscape images to obtain attributes such as road traffic; collecting map images, tiles and vector data to construct a road network topology and attribute database; associating webpage texts, pictures, streetscape images and network map multi-source data according to the spatial position of the traffic facility; through comprehensive and deep attribute mining, different modal data are integrated to improve the accuracy and reliability of attribute mining, the real-time and dynamic updating capability, the convenient visualization and interaction operation, the data sharing and integration convenience, and the traffic facility management efficiency and collaboration are improved. According to the method, comprehensive, accurate and real-time traffic facility attribute data support can be provided for urban traffic planning, traffic management and intelligent traffic system construction, so that the efficiency and the intelligent level of traffic facility management are remarkably improved.

The invention provides an attack scene reconstruction method based on network threat clue causal mining, and relates to the technical field of network security. The method comprises the following steps: performing feature extraction and representation on collected network data to obtain multi-source data; extracting open source data based on an open source threat intelligence platform to establish a marginal causal relationship of multi-source data; the method comprises the following steps: constructing a completely undirected graph based on multi-source data, initializing a condition set, testing whether adjacent node pairs are independent or not based on the condition set, removing edges without a direct causal relationship when the adjacent node pairs are independent, obtaining a skeleton graph, and expanding a marginal causal relationship based on the condition set at the moment; and performing edge orientation processing on the skeleton diagram based on the causal priori knowledge base to obtain a causal diagram, and obtaining a causal relationship between the network data based on the causal diagram and reconstructing an attack path. According to the method, a causal discovery process based on marginal causal priori knowledge extension is introduced, so that the generalization ability of the model is enhanced, and the adaptability to unknown attacks is improved.

Disclosed in the present invention is a wetland ecological resilience three-dimensional evaluation method, comprising: acquiring wetland ecological original data; on the basis of the original data, acquiring three-dimensional evaluation data; processing the three-dimensional evaluation data to obtain a wetland ecological resilience value; and, on the basis of the wetland ecological resilience value, evaluating wetland ecological resilience. The present invention can comprehensively reveal ecological resilience levels of wetlands. In addition, wetland data, vector data and statistical data are all open source data, for which the difficulty in acquiring is low, The evaluation method has a wide range of application and low operating costs, is easy and convenient to operate, and has high precision of wetland ecological resilience evaluation.

The invention provides a domain reasoning model training method and device, a domain reasoning model using method and device, equipment and a medium, and relates to the artificial intelligence technology. The domain inference model training method comprises the steps that an original domain model is finely adjusted based on a preset inference data set, and an intermediate model with the inference capability is obtained; the preset reasoning data set is generated by reasoning general open source data through a reasoning model and comprises reasoning process data and reasoning conclusion data which conform to the output form of the target domain; based on the current inference data set, performing iterative training on the basis of the original domain model to obtain a domain inference model; wherein in each iteration process, the current reasoning data set is updated based on the field problem data set and a current intermediate model obtained based on the current reasoning data set in the current iteration process; the domain problem data set comprises task feature information of the target domain. By means of the method, the domain model reasoning effect can be effectively improved.

The invention relates to the technical field of meteorological data management, and particularly discloses a meteorological data management system and method based on multi-source data collaboration, and the system comprises an automatic open source data downloading module, an automatic sunflower satellite ground data receiving and processing module, and an automatic satellite cloud picture shooting planning module. The OBS storage module is used for storing information and is connected with other modules; the satellite cloud picture data automatic processing module is used for inquiring corresponding meteorological information in the OBS storage module according to the information in the cloud picture data, and automatically assigning and filing meteorological elements; and the satellite cloud picture data automatic processing module is connected with the satellite cloud picture shooting automatic planning module. According to the invention, advanced technical achievements in the fields of meteorology, information technology, database technology, data processing and the like are deeply fused, and a highly integrated and intelligent meteorological data management platform is constructed.

The invention discloses a mechanical drilling rate prediction method of a bidirectional LSTM model based on attention enhancement, and the method comprises the steps: selecting an open source data set from a Norway Volve oil field, carrying out the preprocessing, and enabling all data to be logging data based on a depth domain; establishing a mechanical drilling speed prediction model by using an attention-based bidirectional LSTM network, and training the model to obtain an optimized mechanical drilling speed prediction model; according to the invention, by introducing an attention mechanism, the model can automatically identify and preferentially process data which has great influence on a prediction result, so that the prediction accuracy and efficiency are remarkably improved. Compared with a traditional one-way LSTM, the two-way LSTM can consider past and future data information at the same time, and a more effective solution is provided for processing time series data with complex nonlinear characteristics.