223 results about "Open source data" patented technology

The invention discloses a knowledge graph analysis and question answering method and device based on a large language model. The method comprises the steps of obtaining original data, performing data processing based on a large language model to obtain target data, constructing a knowledge graph according to the target data, and storing the knowledge graph into a graph database; acquiring an open source data set, and processing the open source data set by using the large language model to obtain a training data set; according to the graph database, constructing a dictionary which is used for reflecting a graph structure of the graph database; utilizing the training data set, the dictionary and a preset prompt instruction to finely adjust the large language model to obtain an optimized large model; generating a query statement by utilizing the graph database and the optimization large model according to a query problem of a user, and determining a query result in the graph database according to the query statement; and based on the optimization large model, carrying out integration processing on the query question and the query result to obtain a final answer. According to the method, the construction process of the knowledge graph is simplified, and the accuracy of analysis and question answering is improved.

A cooling system running method includes obtaining an open source data set of a cooling system including components each being configured with parameter(s). The open source data set includes a parameter value corresponding to each parameter, and the parameter(s) configured for one component include a controllable running parameter. The method further includes determining an initial model for predicting power consumption of the cooling system, selecting a target algorithm using an open toolbox, training the initial model using the open source data set and the target algorithm to obtain a target model, deploying the target model to a controller of the cooling system, and causing the controller to at least control operation of the one component according to a parameter value corresponding to the controllable running parameter that is defined in a power consumption optimization policy determined by the controller using the target model.

The invention discloses a strategy question-answering system and method based on a vertical domain large model, and the method comprises the steps: S1, receiving the natural language input of a user, and retrieving an open source data set to obtain an original data set with the comprehensive similarity meeting the requirements; s2, performing necessary data cleaning and preprocessing on the original data set to obtain a training set; s3, performing fine tuning on the pre-processed basic large language model by adopting the training data set, and introducing a low-rank structure to modify a weight matrix to obtain a final model; s4, the problem enters a final model for post-processing to generate prediction output; the system takes a large language basic model subjected to fine tuning training as a core, introduces a low-rank structure to perform fine tuning on model parameters, reduces parameter quantity required by training while keeping model performance, reduces model complexity, reduces an overfitting risk, improves generalization ability, supports deep fusion of a specific field knowledge base and universal field data, and has a wide application prospect. And the accuracy and correlation of questions and answers are improved.

The invention discloses an intelligent content generation system based on open source data analysis, and relates to the technical field of monitoring analysis, and the system comprises a data preprocessing module which is used for carrying out the multi-modal content collection of a heterogeneous open source data platform, and obtaining a structured semantic feature matrix; the graph construction and extraction module is used for constructing a domain knowledge graph based on the structured semantic feature matrix to obtain a domain evolution feature vector; the intelligent content generation module is used for inputting the domain evolution feature vector into a multi-modal generative adversarial network for semantic-visual cross-modal mapping to obtain style-enhanced intelligent generation content; and the optimization module is used for performing propagation effect simulation prediction on the style-enhanced intelligent generation content, generating a content propagation influence matrix, performing optimization iteration on the intelligent generation content according to the content propagation influence matrix, generating final optimization content and pushing the final optimization content to a target terminal.

The invention relates to the technical field of risk management, in particular to a multi-scale urban rainstorm flood risk assessment method. The method comprises the following steps: acquiring a regional basic data set; performing data preprocessing on the regional basic data set to generate a regional basic preprocessed data set; constructing a rainstorm flood risk assessment index system based on the regional basic preprocessing data set to obtain an urban scale flood assessment index, and performing urban scale rainstorm flood risk assessment to obtain an urban scale rainstorm flood risk assessment result; according to the method, the index information is obtained from the open source data, and the defects that in previous researches, part of index data is poor in spatial precision and long in time are overcome; the multi-scale urban rainstorm flood risk assessment method comprehensively considering the urban scale and the block scale is provided, and the accuracy, scientificity and comprehensiveness of urban rainstorm flood risk assessment can be improved.

The invention relates to a method for converting Lao characters into phonemes based on transfer learning, and aims to realize efficient phoneme conversion by utilizing a model pre-trained on other languages and finely adjusting limited Lao data. According to the method, the problem of data scarcity can be relieved, the training process of the model can be accelerated, and the generalization ability of the model on new tasks is improved. A more robust Lao character transphoneme model can be constructed through transfer learning, and a foundation is laid for subsequent speech synthesis. As Thai with high language similarity with Lao is also a low-resource language and cannot be used as a pre-training language, Chinese is selected as the pre-training language. A Chinese open source data set is firstly used for training a Chinese G2P model based on a Transform architecture, decoding end parameters of the model are initialized to a student model, fine tuning training is performed by using a Lao data set with the scale of ten thousand, and the accuracy of a Lao character-to-phoneme model is improved.

The present application provides a translation model training method, translation method, device, electronic equipment, and medium, relating to the field of translation model technology. The translation model training method includes: expanding vocabulary of the translation model with commonly used vocabulary and specialized vocabulary in advance, optimizing feature vectors in the vocabulary based on a large language model, cleaning an open-source dataset using a data cleaning method to filter and retain corpora meeting a quality standard, and training the translation model using filtered corpora. The translation method includes: obtaining source language text to be translated, inputting the source language text into a translation model to obtain target language text after translation. The embodiments of this application not only enhance the vocabulary of the translation model but also improve the accuracy of the translation model in specialized domains.

The invention discloses a recommendation system deviation correction method fusing big language model world knowledge, which comprises the following steps: acquiring an open source data set of a recommendation system, and generating a training set; guiding a large language model to generate description information of an article and preference reasoning of a user through a training set in combination with a context learning technology; encoding the description information of the article and the preference reasoning of the user through a pre-trained text encoder to generate a first encoding vector, and encoding the IDs of the user and the article to generate a second encoding vector; inputting the first coding vector and the second coding vector into a multi-mode expert network module to obtain final representations of the user and the article; inputting the final representations of the user and the article into a prediction layer to obtain a prediction value, and optimizing a multi-modal expert network module through a cross entropy loss function; and performing recommendation system correction through the optimized multi-mode expert network module. The inherent deviation in the user behavior data is eliminated, the real preference of the user is captured, and recommendation system deviation correction is completed.

The invention discloses a hybrid photovoltaic prediction method based on SARIMA-LSTM, and the method comprises the steps: carrying out the calculation according to historical photovoltaic power generation data in an SCADA system, wherein the historical photovoltaic power generation data comprise the power generation power and the accumulated power generation amount, and the meteorological data comprise the irradiance, the temperature, the humidity, the wind speed, the cloud cover and the like, and are obtained from a meteorological station or an open-source data platform; statistics is carried out by combining an SARIMA model with a seasonal prediction function and an LSTM model for processing a time sequence, and a more accurate photovoltaic power generation prediction value is obtained after combined prediction and evaluation. Compared with the prior art, short-term prediction and long-term prediction can be considered, and seasonal change data prediction is more accurate.

A method for evaluating an artificial intelligence (AI) large language model (LLM) generated response, the method comprising receiving a user query in the form of patient-related questions for a medical treatment domain; analyzing the user query using a LLM learned with open-source data and outputting a LLM answer from the LLM; performing a domain-specific evaluation of the LLM answer; performing a domain-agnostic evaluation of the LLM answer; generating at least one metric for the LLM answer based on the domain-specific evaluation and the domain-agnostic evaluation of the LLM answer; and evaluating the quality of the LLM answer based on the at least one metric.

The invention belongs to the technical field of extreme ocean phenomenon forecasting, and discloses a typhoon and storm surge lightweight rapid forecasting system and method. The method comprises the following steps: preprocessing a multi-source marine environment data set and a typhoon moving path; constructing a typhoon and storm surge data set based on the preprocessed data; based on the constructed typhoon and storm surge data set, establishing a deep learning model VIT-TSS based on VIT; and comparing the output result of the deep learning model VIT-TSS with the open source data and the measured data to verify the forecast precision. The typhoon and storm surge intelligent forecasting model provided by the invention can realize typhoon path forecasting and regional water increase forecasting. According to the VIT-TSS model provided by the invention, the prediction time of the model is greatly shortened while the calculation precision is improved. The system provides key information for emergency management and navigation.

The invention discloses a flight service system based on a knowledge graph and a large model. Comprising an ontology construction module used for predefining related information of a flight service system based on user demands; the data acquisition module is used for acquiring open source data related to the flight service system from multiple channels and performing deduplication and normalization processing on the open source data to obtain standard open source data; the knowledge graph construction module is used for extracting triads from standard open source data based on the related information to obtain a triad set of the knowledge graph; the prompt generation module is used for generating a corresponding target prompt based on a user demand; the large model fine tuning module is used for training a pre-constructed large model based on the target prompt and the triple set to obtain a trained large model; the trained large model is used for generating a result matched with the target task according to an input user prompt and the triple. According to the invention, diversified services can be provided for users.

The invention relates to the technical field of power grids, in particular to a vertical domain distribution network planning method and system based on a large language model. The method comprises the following steps: collecting and arranging a distribution network structure and operation data, and constructing a'graph 'management system; extracting historical planning schemes to form a typical case library; fusing external open source data and source side data to establish a big database and an access interface; constructing a bilateral prediction model and a planning scheme generation model based on machine learning; carrying out secondary pre-training and practical packaging on the open source large language model; planning parameters are obtained in a guiding mode in a question-answer mode, and a scheme is generated. According to the method, the problems of high labor cost, long planning time, large error, unstable planning quality, poor interactivity, high learning threshold and the like of traditional distribution network planning are solved, the planning efficiency and precision are improved through an intelligent technology, the professional threshold is reduced, a brand new solution is provided for coping with the new situation of high permeability of new energy and electric energy replacement acceleration, and the method is worthy of popularization and application. And the practical value is remarkable.

The invention relates to the technical field of monitoring video processing, in particular to a monitoring video frame identification method based on an image segmentation technology, which comprises the following steps: S1, acquiring a video file: downloading past monitoring videos through a network, and uniformly modifying the monitoring videos into the same format; s2, performing optimization processing on the video: selecting a monitoring video, inputting the monitoring video into optimization software for optimization processing, and improving the definition of an image in the monitoring video; according to the method, a model program can be constructed, the model program performs deep processing on the image through an open source data set in the fields of deep learning and image processing, efficient feature representation can be realized, the accuracy and the fine degree of image semantic segmentation are improved to a great extent, the number of image segmentation is increased, the error segmentation phenomenon is reduced, and the image segmentation efficiency is improved. The phenomena of image edge blurring, wrong segmentation and instable segmentation of a segmentation result generated by a traditional segmentation method are avoided.

The invention relates to the technical field of big data, and discloses a data processing method and device, computer equipment, a storage medium and a program product. The method is applied to a NiFi cluster, and comprises the following steps: acquiring first target data; wherein the first target data is used for indicating changed data; performing type conversion processing on the first target data to obtain second target data; performing field mapping on the second target data to obtain third target data; and writing the third target data into an open-source data lake storage framework in a java client mode.

The invention relates to a power battery and energy storage battery health state prediction method and system, and the method comprises the following steps: S1, obtaining original EIS data through an open source data set, extracting the center position, radius and arc length of a medium-high frequency Nyquist diagram arc, and extracting the length of a low-frequency Nyquist diagram as health features, and obtaining a training set; step S2, constructing a DGPR model; inputting the training set into the DGPR for training, selecting a kernel function, training to obtain a hyper-parameter, and establishing an accurate battery health state prediction model; and S3, inputting the health features extracted in the test set into the trained health state estimation model, and performing battery health state prediction according to hyper-parameters obtained by training. According to the method, the health features can be extracted by intercepting the EIS frequency band data, the health features are input into the DGPR for training to obtain the hyper-parameters, and the health state of the battery is predicted according to the hyper-parameters.

The invention discloses a field data collection method and device, electronic equipment and a storage medium, and relates to the technical field of data processing, and the method comprises the steps: obtaining an open source data set in an open source corpus, and carrying out the field correlation labeling of all data in the open source data set, and obtaining a labeled data set; screening the annotated data set according to the annotated domain correlation to obtain an annotated data subset related to the target domain; performing compliance annotation on each piece of data in the annotation data subset to obtain an annotation data subset; and screening the annotation data subset according to the compliance of the annotation to obtain target domain data. According to the method, professional corpus data which not only has financial field correlation but also meets compliance requirements is efficiently extracted from a large amount of open source corpora, the quantity and quality of data in the target field are ensured, and a reliable data basis is provided for pre-training of a large model in the financial field; therefore, the understanding ability and expression accuracy of the model in the financial field are improved.

The invention discloses a satellite network anomaly detection method based on federated learning, relates to the technical field of satellite network communication, and solves the problems that an existing detection system cannot effectively learn a feature model between data and cannot realize balance of optimal model training and communication overhead. According to the method, an open source data set is preprocessed, a CvT-LSTM model is constructed, a general framework with a single server and K participants is designed, an aggregation strategy based on local data volume and anomaly detection model precision weighting is used, anomaly detection model parameters of a selected client are uploaded to a global model, and after being aggregated by the global model, the anomaly detection model parameters of the selected client are obtained. And issuing to all clients, and the like. According to the method, interference of low-quality data is avoided, transmission and calculation of invalid data are reduced from the source, the communication cost is remarkably reduced, only the high-representativeness client participates in global model aggregation, and the communication burden is reduced to the maximum extent while the model performance is ensured.

The invention discloses an anti-cancer drug sensitivity prediction method and system based on bipartite graph and hypergraph learning, and belongs to the technical field of computer software. The method comprises the following steps: acquiring cell line and drug data and drug and cell line reaction original IC50 values by using an open source database, and standardizing the IC50 values; constructing a bipartite graph of the cell line and the drug, and obtaining embedded representation of the drug and the cell line by applying a graph convolutional network; super nodes are introduced to construct a cell line-drug isomerism hypergraph, and node features are learned through GCN; and finally, fusing the output of the GCN layer by applying an attention mechanism, and predicting the final association probability of the cell line-drug pair in combination with a full-connection layer. According to the method, combined exploration and collaborative optimization of the bipartite graph and the hypergraph are achieved, meanwhile, potential and complex relations between the medicine and the cell line are explored through the hypergraph, the quality of node feature representation is jointly improved, and therefore the association score between the medicine and the cell line is predicted more accurately.

The invention discloses a clothing attribute identification method and system, computer equipment and a readable storage medium, and relates to the technical field of image identification. The clothing attribute identification method comprises the following steps: acquiring a to-be-identified picture; and obtaining a picture text database of the clothes, and obtaining a clothes attribute recognition model according to the picture text database of the clothes and the deep learning model. And according to the clothing attribute identification model, identifying the attribute of the clothing in the to-be-identified picture, and obtaining the attribute data of the clothing in the to-be-identified picture. In the aspect of data, massive clothing data is crawled through a crawler technology, part of open source data sets are combined, pre-labeling is performed by means of the powerful content understanding ability of a visual language large model, and a professional clothing attribute data set is constructed. In the algorithm aspect, the advantages of a Transform architecture and a CNN are combined, a lightweight network structure with a self-attention mechanism based on a local sliding frame and key point regression as an auxiliary task is designed, and the method is used for solving the problem of clothing multi-attribute recognition and giving consideration to model precision and reasoning time delay.

The invention discloses a small target detection method under an unmanned aerial vehicle visual angle based on an SCGD-YOLO network, and the method comprises the following steps: obtaining an open source data set of a small target under the unmanned aerial vehicle visual angle, dividing the data set into a training set, a verification set and a test set, the data set comprising ten types, and the main types being small targets; configuring a network environment required by the network model; constructing an SCGD-YOLO network model, wherein the SCGD-YOLO network model comprises a backbone network, a neck network and a detection head; sending pictures and labels of the training data set into the constructed SCGD-YOLO network model for training, and adjusting corresponding hyper-parameters according to a result of the verification set to obtain an optimal training result; finally, to-be-detected pictures in the test set are sent into the trained SCGD-YOLO network model for small target detection, and a detection result is output, the problems that the small target resolution is low and the detection precision is reduced when the unmanned aerial vehicle carries out high-altitude shooting are solved, the detection precision of the small target is improved, and the parameter quantity of the model is reduced.

The invention relates to a security alarm noise reduction method based on rules and a large model, which belongs to the field of network security and comprises the following steps of: collecting security alarm data and security logs and preprocessing the security alarm data and the security logs; constructing a dynamic comprehensive rule knowledge base comprising a screening rule base and a plug-in knowledge base, and periodically updating the dynamic comprehensive rule knowledge base; selecting an open-source large model of the security field, using the open-source data set of the security field and historical security alarm data, and utilizing a distillation technology to finely adjust a noise reduction small model; a knowledge base driven periodic fine tuning and dynamic knowledge injection double-path coordination mechanism is adopted to improve the capability of the noise reduction small model; and finally, realizing efficient and accurate alarm noise reduction based on the screening rule base and the noise reduction small model. According to the invention, the efficiency and accuracy of security alarm noise reduction are effectively improved, and an efficient, accurate and intelligent alarm noise reduction method is provided for the field of network security.

The invention provides a data processing method and system, electronic equipment, a storage medium and a product, and relates to the technical field of information. The data processing method comprises the following steps: collecting original reconciliation data of each business department, taking a distributed file system as bottom storage, constructing a data warehouse framework and a distributed column-oriented open source database on the basis, and managing the original reconciliation data. Based on a user query instruction, target account checking data of a current checking time period is obtained from an open source database, and the data relates to more than two business departments. And then target reconciliation data are verified, and data of different departments are checked in a cross manner. According to the scheme, a large amount of data is efficiently stored through the distributed file system, ordered management and efficient analysis are achieved through an upper-layer framework and a database, quick and accurate response query can be achieved, the data accuracy and reliability are improved, potential business problems can be perceived in time, and business promotion and financial accounting accuracy are guaranteed.

The invention provides a monocular multi-target tracking and speed measuring method based on deep learning, and the method comprises the following steps: preparing a training data set, collecting the data of real vehicles and pedestrians in real time through a camera under a traffic real scene, adding an open source data set, and carrying out the real-time collection of the data; the method can effectively improve the robustness and generalization of subsequent reasoning of a model, adopts YOLOv5 as a backbone network architecture for model training, trains the vehicle and pedestrian data sets obtained in the S1 to obtain a target detection model, and uses the vehicle and pedestrian target detection model trained in the S2 to quickly and accurately detect all traffic objects in a video frame. And then the detection results are transmitted to a tracking algorithm, and the movement speed of the traffic object is measured and calculated. The monocular multi-target tracking and speed measuring method based on deep learning provided by the invention is high in operation speed in practical application reasoning, very high in adaptability in hardware deployment, simple and small in model packaging, and low in hardware computing power performance requirement.

The invention discloses a method for automatically producing whole-body digital human data, which comprises a first model training stage: collecting a real human speaking data set and acquiring an open source data set as an initial training data set, training a lip shape alignment model and a digital human generation model, and performing data enhancement and data expansion processing based on the initial training data set to obtain a second model training stage; continuing to train the lip shape alignment model and the digital human generation model until the precision of the lip shape alignment model and the digital human generation model meets a preset model parameter threshold value; and a second model training stage: when the data volume of the training data set in the first model training stage reaches a preset data volume threshold value, generating new description text data based on the video data in the training data set, and adding the description text data into the video data set, performing fine tuning on the digital human generation model and replacing the corresponding model in the first model training stage; and executing loop iteration of the first model training stage and the second model training stage at the same time until a preset training target is met.

The invention provides a multi-factor stress-considered regional trafficability potential evaluation method, which comprises the following steps of: 1) extracting topographic elements, and performing topographic complexity inversion by combining optical and topographic data; 2) collecting existing open source data to obtain national road network information, and performing inversion on surface cover type information; 3) evaluating an area with extreme temperature and rainfall based on historical time sequence climate data, and performing inversion of medium-high resolution soil moisture content information by using deep learning and multi-source remote sensing data based on a kilometer-level soil moisture content product of an earth surface cover type and an open source; 4) carrying out traffic safety factor inversion; and 5) for comprehensive analysis of each factor, carrying out passable potential evaluation based on an analytic hierarchy process. The method effectively solves the problems that in the regional trafficability potential evaluation process, geology and environment influence factors are not fully considered, related remote sensing element recognition precision under the wide area scale is low, efficiency is insufficient, and the generalization ability of an intelligent method is poor.

An instance segmentation model training method includes training the instance segmentation model firstly based on a first data set stored in a first database, and training the firstly-trained instance segmentation model secondly based on a second dataset stored in a second database, where the second dataset includes a large-scale open-source dataset and a segmentation target object-absent image acquired by capturing a working environment within an industrial site.

The invention relates to the technical field of bioinformatics, and provides a CDA prediction model composed of an original channel and an interaction channel. The method comprises the following steps: firstly, acquiring original data from two open source databases of circ2Disease and circR2Disease; the method comprises the following steps: processing original data, screening out human circRNA and disease association, constructing a sparse matrix, carrying out multiple similarity calculations based on the sparse matrix to obtain initial features, and then entering two channels: extracting high-order features and retaining important information by using a double-attention mechanism in a channel I; in the second channel, degree segmentation and Wave modules are used, and potential information association of different sample features is fused. And then, the information of the two channels is integrated through a channel fusion module to generate final output, and finally, a model with optimal performance is found through five-fold cross validation and evaluation indexes. Experimental results based on two data sets show that the performance of the model in a CDA prediction task is superior to that of an existing method, and further case research and generalization analysis verify the effectiveness and wide applicability of the model.

The invention relates to the technical field of monitoring video processing, in particular to a monitoring video frame identification method based on an image segmentation technology, which comprises the following steps: S1, acquiring a video file: downloading past monitoring videos through a network, and uniformly modifying the monitoring videos into the same format; s2, performing optimization processing on the video: selecting a monitoring video, inputting the monitoring video into optimization software for optimization processing, and improving the definition of an image in the monitoring video; according to the method, a model program can be constructed, the model program performs deep processing on the image through an open source data set in the fields of deep learning and image processing, efficient feature representation can be realized, the accuracy and the fine degree of image semantic segmentation are improved to a great extent, the number of image segmentation is increased, the error segmentation phenomenon is reduced, and the image segmentation efficiency is improved. The phenomena of image edge blurring, wrong segmentation and instable segmentation of a segmentation result generated by a traditional segmentation method are avoided.

The invention relates to the technical field of data processing, and discloses a quotation analysis model construction method and device, equipment and a medium, and the method comprises the steps: obtaining original full-amount data, and labeling the original full-amount data according to a preset structured format to obtain initial full-amount data; wherein the original total data represents complete data including historical quotation data and newly added quotation data on that day; cleaning dirty data based on the initial total data to obtain processed total data; wherein the dirty data cleaning comprises the step of carrying out outlier data screening by utilizing a trained instruction fine tuning model; determining open source data as unstructured data, and comprehensively processing the unstructured data and the processed total data to obtain target training data; and training a large language model by using the target training data to obtain a quotation analysis model. The method has the beneficial effects of standard structured text and real-time response.