49 results about "Open source data" patented technology

The invention relates to a security alarm noise reduction method based on rules and a large model, which belongs to the field of network security and comprises the following steps of: collecting security alarm data and security logs and preprocessing the security alarm data and the security logs; constructing a dynamic comprehensive rule knowledge base comprising a screening rule base and a plug-in knowledge base, and periodically updating the dynamic comprehensive rule knowledge base; selecting an open-source large model of the security field, using the open-source data set of the security field and historical security alarm data, and utilizing a distillation technology to finely adjust a noise reduction small model; a knowledge base driven periodic fine tuning and dynamic knowledge injection double-path coordination mechanism is adopted to improve the capability of the noise reduction small model; and finally, realizing efficient and accurate alarm noise reduction based on the screening rule base and the noise reduction small model. According to the invention, the efficiency and accuracy of security alarm noise reduction are effectively improved, and an efficient, accurate and intelligent alarm noise reduction method is provided for the field of network security.

The invention provides a domain reasoning model training method and device, a domain reasoning model using method and device, equipment and a medium, and relates to the artificial intelligence technology. The domain inference model training method comprises the steps that an original domain model is finely adjusted based on a preset inference data set, and an intermediate model with the inference capability is obtained; the preset reasoning data set is generated by reasoning general open source data through a reasoning model and comprises reasoning process data and reasoning conclusion data which conform to the output form of the target domain; based on the current inference data set, performing iterative training on the basis of the original domain model to obtain a domain inference model; wherein in each iteration process, the current reasoning data set is updated based on the field problem data set and a current intermediate model obtained based on the current reasoning data set in the current iteration process; the domain problem data set comprises task feature information of the target domain. By means of the method, the domain model reasoning effect can be effectively improved.

The invention discloses a method for data mutual migration between relational databases, which is characterized by comprising the following steps that: an SQLite database is opened, a user sets an original password by himself / herself, and the SQLite database is an open source SQLite database; a structural body is self-defined in codes of the SQLite database and used for configuring parameters and achieving encryption of the SQLite database, and the structural body comprises a database original password, an original password byte length, a tree structure, an encryption password generated after the database original password is adopted for encoding and a decryption password generated after the database original password is adopted for encoding; generating an encrypted password and a decrypted password by adopting the original password; data writing of the SQLite data the encrypted password and to-be-written data are subjected to xor according to word bits, and the result serves as encrypted data to be written into the SQLite database; and reading the data of the SQLite database, wherein the decryption password and the data to be read are subjected to xor according to word bits, and a result is used as a read result of the decrypted data of the SQLite database.

The application discloses a kind of open source data semi-automatic data labeling method and system based on multi-modal large model, belong to data labeling technical field.The method includes: enhancement processing;Determine image-text-category data pair training set and image-text-category data pair test set;Adjust SimCSE text encoder parameter;Adjust CLIP-SimCSE model parameter;Adjust CLIP-ViT model parameter;Test in multi-modal large model, to adjust multi-modal large model parameter;Classified labeling is carried out to determine the different image-text data pairs in the labeling result of CLIP-SimCSE model and CLIP-ViT model;Different image-text data pairs are corrected.The application can guarantee the accuracy of professional field data category labeling result, improve data labeling efficiency, significantly reduce the cost of artificial data verification.

The invention provides an ionosphere parameter and short wave frequency band prediction method and electronic equipment, and the method comprises the steps: obtaining the ionosphere parameter data in a corresponding IRI model above the midpoint of a great circle through obtaining the geographic coordinates of two places of a communication link and the midpoint of the great circle, obtaining the geomagnetic solar activity data of an open-source data website, and obtaining the geomagnetic solar activity data of the open-source data website; the method comprises the following steps: carrying out ionosphere oblique detection by using ionosphere detectors at two communication places, obtaining an ionosphere oblique detection ionogram, obtaining frequency band data available for short-wave communication from the oblique detection ionogram, carrying out data preprocessing, selecting different tasks, setting corresponding Transform-LSTM model hyper-parameters, carrying out model training, and storing optimal model parameters; and performing an ionosphere parameter prediction task or a short-wave communication available frequency band prediction task by using the corresponding model. The ionosphere detection data, the environment monitoring data and the like are fused in the deep learning field and the short-wave communication field, and the method has important significance and application prospects.

The application relates to a corner case-based deployment model construction method, which comprises the following steps: obtaining a first data set according to automatic driving images and preset open source data, obtaining a second data set according to the automatic driving images, the preset open source data and part of image in a corner case scene image, obtaining a third data set according to the remaining images in the corner case scene image, and merging the second data set and the third data set to obtain a fourth data set; training a preset multi-task detection model based on the fourth data set to obtain an automatic driving perception detection pre-training model, training the pre-training model based on the first data set and a preset period of cosine annealing learning strategy to obtain a plurality of perception detection models; and performing transfer training on the plurality of perception detection models based on the third data set, performing average weight and correction BN on the plurality of perception detection models after the transfer training to obtain a final deployment model.

The invention discloses a code change readability evaluation method based on fine-grained difference perception and enhanced context, and belongs to the crossing field of software engineering and deep learning. The invention constructs a core method for context attention enhancement based on multi-granularity difference representation and difference guidance. Through multi-granularity feature extraction fusing a token level (semantic grammar core change) and a character level (structure details and fine adjustment information), and in combination with a triple context enhancement mechanism of global attention, local attention and similarity attention, accurate judgment of readability change trends (increase / decrease) before and after code change is realized. Experiments on a GitHub open source data set show that the method is remarkably superior to an existing baseline model, readability quality control in a dynamic code evolution scene (such as GitHub warehouse iteration) is effectively supported, and technical support is provided for software maintenance and team cooperation.

The application relates to a case data statistical analysis method for the judicial field, and belongs to the technical field of computer big data. The application organizes judicial case data by using a wide table model in the field of data warehouses, finally forms a case data wide table model and a personnel data wide table model, adopts an open source data calculation and storage framework ClickHouse calculation engine, adopts a dynamic template method to define a statistical analysis model, when statistical analysis is performed, an analysis model template is combined with analysis parameters to dynamically and timely generate a corresponding analysis script, the analysis script is submitted to the calculation engine, calculation and analysis are completed by the calculation engine, and the calculation result is normalized and then transmitted back to a calling party. The overall design idea of the application greatly improves the flexibility of judicial case data in the analysis process.

The invention discloses an urban flood risk assessment method and device based on open source data, and belongs to the technical field of urban flood. The method comprises the following steps: obtaining open source road data, simplifying the open source road data into a drainage pipe network initial topological graph, determining a water outlet, and further carrying out iterative optimization to obtain a drainage pipe network model; and predicting the drainage effect of an actual urban drainage system according to the drainage pipe network model, and evaluating the urban flood risk. The method completely depends on open source data, and thoroughly gets rid of dependence on special drainage pipe network data which is difficult to obtain; according to the method, the modeling efficiency is improved, and through reasonable physical rules and optimization algorithms, the generated model shows good consistency with a real model in a system level (such as total overflow and water outlet flow) and a submerging range, and is sufficient to support planning-level risk assessment requirements.

The invention discloses a track fusion method based on open source data and AIS data, and relates to the technical field of Internet open source data processing and situation fusion. An acquired AIS original message is analyzed to generate a track of a ship target, open source text data acquired through the Internet also includes track information of the ship target, entity extraction is performed by adopting a deep neural network algorithm, the open source text data is processed and recognized, and the track information of the ship target is acquired. Information such as an IMO number, an MMSI number and a ship name of a ship in the data and information such as occurrence time and occurrence place of a related event are identified, ship information and event information are integrated to form a trace point of the ship, and the mark, time and space information of the ship correspond to records in an AIS original message, so that the information of the ship is obtained. And inserting the ship trace point obtained from the open source text data into the track of the ship in the AIS, and further smoothing the track through a Kalman filtering algorithm to form track information after ship target fusion, thereby realizing track fusion based on the open source data and the AIS data.

This invention discloses an environmentally friendly insulating gas molecule and its high-throughput design method and related apparatus, belonging to the field of novel environmentally friendly insulating gas molecule design technology. By establishing a dedicated database containing microscopic discharge parameters and macroscopic physical property parameters of insulating gas molecules, a gas molecule performance evaluation and prediction model is constructed using this database. A molecular structure gene scoring system is established, dominant genes are extracted and recombinated, and chemical space for high-throughput screening is increased by adding groups to the backbone. The performance evaluation and prediction model is used to perform high-throughput screening on a massive number of molecules in an open-source database and molecules generated by adding groups to the backbone. Based on the target property range, a novel environmentally friendly insulating gas molecule is finally obtained. This invention can quickly and accurately evaluate and predict the performance of gases, improve high-throughput screening capabilities, save significant manpower, material resources, and time, and ultimately obtain a novel environmentally friendly insulating gas molecule.

The invention belongs to the technical field of database performance evaluation, and discloses a query performance evaluation method and system for a multi-model database. The method comprises the following steps of: constructing a test data set coexisting with four models, namely a relationship, a document, a graph and a vector, by an open-source data source and a native model, and performing global entity ID semantic association; four types of cross-model query workloads are defined, point query, aggregation, vector ANN and graph matching scenes are covered, and the method is used for systematically evaluating the comprehensive capacity of the multi-model database in the aspects of connection sequence selection, model data conversion cost estimation, aggregation operation optimization and the like. During testing, under the conditions of fixed hardware, a memory buffer area and concurrent parameters, the workload is automatically executed, the median execution time is collected, then the performance score of each load is calculated based on a logarithm normalization speed-up ratio model, and fair quantitative comparison among different multi-model databases is achieved. The method is reproduced in a common multi-model database system to verify the feasibility of the evaluation benchmark.

The embodiment of the invention provides a model training method and device and a related product, and relates to the technical field of machine learning. According to the model training method, the data highly related to the first sample data of the target task can be selected from the open source data set as the second sample data, so that the data redundancy is reduced, the data quality is improved, and the training effect of the model is enhanced; and the most suitable open source model is selected from the open source model library as a sub-model for constructing the hybrid expert model, so that the accuracy of the hybrid expert model is improved. And finally, inputting the first sample data and the second sample data into the hybrid expert model, so that the sub-model which is most matched with the first sample data and the second sample data can output the most accurate execution result, thereby improving the accuracy of the execution result output by the hybrid expert model, and improving the generalization ability of the hybrid expert model.

The invention belongs to the technical field of data processing, and particularly relates to an open source data information mining method and system. The open source data information mining method comprises the steps that multi-modal data of an open source scene are obtained, and the multi-modal data comprise texts, images and time sequence data; performing association confidence coefficient cleaning processing on the data of the open source scene to obtain multi-modal cleaning data; extracting each modal feature from the multi-modal cleaning data, and combining and embedding spatial mapping to obtain a multi-modal semantic vector; according to an attention weight mechanism, performing dynamic feature fusion on the multi-modal semantic vector to obtain a multi-modal fusion feature; and filtering conflict modal information in the multi-modal fusion features, and storing complementary modal information to obtain open source data information mining data.

The invention provides a dam crack defect detection model training method and device based on meta-learning, and relates to the technical field of hydropower engineering.The method comprises the steps that A, a data set is established, the data set comprises a base class data set and a target data set, the base class data set is a similar open source data set, and the target data set is an annotation data set collected by a target dam; b, meta-training is executed, the base class data set is used for conducting meta-training on a defect detection model, and the defect detection model comprises a meta-feature learning device, a feature weighting module and a prediction output module; and the meta-training is carried out by repeatedly executing the following operations: C, replacing the base class data set with the target data set, repeating the steps B01 to B06, and carrying out fine adjustment on the defect detection model to obtain a final defect detection model. According to the method, the bottleneck problem that a deep learning model seriously depends on a large-scale labeled data set when being applied to dam defect detection is effectively solved.

The application provides a kind of combined timing feature information's countermeasure patch generation optimization method, system and storage medium, method includes: step S1, data collection and loading: open source data set, simulation data set, custom real scene data set are loaded, obtain total loading frame number;Step S2, calculate countermeasure patch position: initialize countermeasure patch parameter, read the data of target frame, obtain the label parameter information provided by open source data set, simulation data set, custom real scene data set, generate countermeasure image;Step S3, combine historical BEV information batch optimization: adopt multi-frame timing inference setting, calculate the total loss of continuous multi-frame output result, based on total loss back propagation and gradient optimization, iteration generates final countermeasure patch.The beneficial effects of the present application are: the method of the present application combines the timing BEV feature information provided by the countermeasure patch with the optimization method of the traditional countermeasure patch, and better optimization effect is obtained.

The present application relates to a kind of strong wind-flood multi-disaster under urban refuge place function loss evaluation system, belong to urban resilience and disaster prevention and mitigation technical field, wherein basic data processing module includes network open source data, topology and data processing system, for building city road network-shelter function network topology structure model and the basic data for disaster physical loss analysis;Monitoring module includes meteorological observation data, for obtaining real-time disaster intensity parameter, such as wind speed, wind direction and rain intensity parameter etc.;Calculation module includes the disaster damage analysis system for calculating the physical damage of city road network, building group and shelter component caused by strong wind and flood disaster and the function loss analysis system for calculating the function loss of city shelter caused by corresponding physical damage, the system established more accurately quantifies the function loss of city shelter under the joint action of strong wind-flood multi-disaster, considering the influence of traffic network.

The embodiment of the invention provides a data set labeling method, electronic equipment, a storage medium and a program product. The method comprises the steps of deploying an open source data set labeling model to a preset container cluster and / or deploying a preset artificial intelligence model to the preset container cluster according to model modification parameters; the effective data set is uploaded to object storage for segmentation and encryption storage, and a to-be-labeled data set is obtained; according to the labeling task information, when it is detected that a labeling mode is model labeling, calling a to-be-labeled data set according to the labeling task information, and calling a preset artificial intelligence model and / or an open source data labeling model from a preset container cluster as a target labeling model; decrypting the data set to be labeled; and inputting the decrypted data set into the target labeling model to generate labeled data, and storing the labeled data into the object storage. According to the method, automation and configurability of the labeling process are achieved, and high flexibility and safety are achieved.

The application belongs to the technical field of natural language processing, and particularly relates to a long-term controllable and optimizable corpus construction method suitable for GPT. Controllable text generation mainly reflects that a segment of the beginning of the answer in the question and answer pair is supplemented, and the words related to the important information in the question are guided to answer the actual expected output, so that controllable text generation is realized, and sensitive questions are evaded in the corpus construction by using an evasive answer and the like. Long-term optimization mainly reflects that a user can continuously exclude related questions in an open source data set according to own needs by using an exclusion method, update own training corpus, and thus complete a task required by the user.

The application provides a kind of urban multi-mode traffic network post-disaster repair strategy efficiency evaluation method and system, belongs to the technical field of urban traffic operation management, constructs multi-mode traffic network structure model using road network and bus network open source data;Using traffic travel demand data to simulate urban traffic distribution, construct multi-mode traffic network function model;Based on multi-mode traffic network function model, according to the spatial distribution characteristics of disaster and post-disaster network repair strategy, construct multi-mode traffic network structure recovery model, evaluate the efficiency of structure recovery;Based on multi-mode traffic network structure recovery model, reload travel demand, construct multi-mode traffic network function recovery model, evaluate the efficiency of function recovery.The application can comprehensively consider various traffic modes and the complex dynamic game between travelers, provide more accurate and more comprehensive repair strategy evaluation, can enhance the resilience of urban integrated transport system, so that it can better cope with various disasters and emergencies.

The application discloses an image inpainting method based on a downsampling confidence network, comprising the following steps: 1) preparing an open source data set and dividing the data set into a training set, a validation set and a test set; 2) designing a downsampling confidence network, specifically taking DeepFillv2 as an image generation module, and proposing a downsampling confidence module, and adding the confidence pixels generated by the downsampling confidence module to the input image of the image generation module; 3) training and verifying the downsampling confidence network by using the training set and the validation set; 4) testing the trained downsampling confidence network by using the test set, and outputting the repaired image. The confidence pixels generated based on the downsampling confidence network obtain the information of a far distance of the image, which helps to strengthen the constraint of the center of the missing area of the output image, achieves a semantic clear result, and not only ensures excellent performance, but also better controls the calculation cost.

This invention discloses a source code summarization method, specifically including the following steps: First, acquire an open-source dataset and preprocess it to generate an Abstract Syntax Tree (AST) and a SimSBT traversal sequence for each code fragment; next, retrieve other code fragments similar to the target code fragment from a corpus as summarization prototypes; then, refine the summarization prototypes using an extractor, retaining key information and removing redundant content; finally, generate the target summary using an editor. This method effectively identifies and corrects redundant and erroneous information in the summary prototypes, and by combining retrieval and generation strategies, the generated summary more accurately reflects the actual function of the code.

The application discloses a ship instance segmentation algorithm based on global and local attention mechanisms, which comprises the following steps: constructing a ship instance segmentation dataset, data enhancement, constructing a SOLOv2 ship instance segmentation network using a global and local attention mechanism GALA, network training and testing, etc. Two new VL ship datasets are collected and manually labeled, and considering the particularity of marine ship instance segmentation, a GALA attention mechanism is proposed, which uses 1D strip pooling and 2D global pooling to save the global and local information of the input image, so as to retain the global position and semantic information of the ship, thereby improving the segmentation accuracy by 4.3% on average compared with existing attention mechanisms. The new open source dataset and the proposed GALA mechanism will promote the research of VL ship applications and attract attention in other computer vision fields to use the GALA mechanism.

The application relates to a rule and large model-based security alarm noise reduction method, belonging to the network security field, which comprises the following steps: collecting security alarm data and security logs and performing pretreatment; constructing a dynamic comprehensive rule knowledge base containing a screening rule base and an external knowledge base, and periodically updating the dynamic comprehensive rule knowledge base; selecting an open source large model in the security field, using a security field open source data set and historical security alarm data, and fine-tuning a noise reduction small model by using a distillation technology; adopting a double-path coordination mechanism of periodic fine-tuning driven by the knowledge base and dynamic knowledge injection to improve the capability of the noise reduction small model; and finally, realizing efficient and accurate alarm noise reduction based on the screening rule base and the noise reduction small model. The application effectively improves the efficiency and accuracy of security alarm noise reduction, and provides an efficient, accurate and intelligent alarm noise reduction method for the network security field.

The invention provides an automatic extraction method, system and device for literature meta-information, and the method comprises the steps: extracting the meta-information, such as a title, an author, a digital object identifier, an abstract and a reference, of a literature by using a natural language name recognition model and a keyword matching model for a structured literature according to keywords and semantic information, and extracting the meta-information, such as the title, the author, the digital object identifier, the abstract and the reference, of the structured literature; meanwhile, Chinese and English literatures are supported; according to the extracted literature meta-information, performing secondary supplement correction by using an online open source database, and constructing a multi-source scheme of literature meta-information extraction; and summarizing and sorting the extracted document meta-information, performing post-processing mapping, and standardizing formats to align to national standards or formats required by users. According to the method, the natural language algorithm and the machine learning algorithm are combined, automatic and efficient literature meta-information extraction is achieved, and particularly when a large amount of literature meta-information query requirements are met, the accuracy and convenience of literature management are improved.

The invention provides a target information mining method and device based on open source data retrieval, and the method comprises the steps: collecting open source data related to a target, and carrying out the preprocessing of the open source data; extracting text features from the preprocessed open source data by using a natural language processing technology; and an improved PSO-K-means clustering algorithm is adopted to perform clustering analysis on the extracted text features to obtain a target information mining result, and the clustering number K in the improved PSO-K-means clustering algorithm can be dynamically adjusted and / or the particle speed is updated by using an adaptive inertia weight. According to the invention, the accuracy and efficiency of information mining are obviously improved.

The invention provides a video clip retrieval method and device, electronic equipment and a readable storage medium, and the method comprises the steps: carrying out the scene segmentation, motion judgment, aesthetic scoring and textbox recognition of an original video based on a plurality of data processing operators, and obtaining a multi-segmentation video; generating a vector database corresponding to the multi-segmented video according to the semantic analysis result of the multi-segmented video; vector retrieval is carried out in the vector database according to the user demand instruction, and a user-defined open source data set is constructed and comprises a plurality of target video clips. According to the method, through multi-dimensional automatic screening and semantic vector retrieval, efficient and automatic generation of a high-quality open source video data set customized as required is realized, the automation level and practicability of data construction are remarkably improved, the specific requirements of different fields and application scenes for data are met, and the method is suitable for large-scale popularization and application. And more accurate and more targeted data support is provided for model training.

The present application relates to the technical field of artificial intelligence decision analysis and behavior modeling, in particular to a kind of situation-reasoning one figure-based character decision mode analysis method and equipment, including the construction decision mode situation-reasoning one figure, the extraction and quantization of decision influencing factors and the extraction and quantization steps of decision influencing factors, in the execution above-mentioned steps, first, the structure of situation-reasoning one figure is used, and massive unstructured open source data is converted into computable, situation-reasoning one figure that can be deduced;Then, according to the decision theory, the internal psychological cognitive characteristics of the character are quantified, and they are used as the key constraint condition of reasoning network;Finally, the decision mode neural network is constructed, and the decision process of "perception stimulation-cognitive screening-logical reasoning" of the character under a certain situation is simulated.The present application realizes the deep integration and accurate analysis from external objective situation to internal subjective cognition.

The invention discloses a spine key point prediction and correction algorithm based on deep learning and biological geometric constraints, which comprises the following steps of: 1, preparing standard spine-lacking open source data sets such as COCO17 and COCO133 as basic data sets, manually labeling spine data in a small batch, and carrying out prediction and correction on spine key points according to the standard spine-lacking open source data sets; the basic data set serves as input data, and the spine key point data set serves as output data. 2, the key point prediction model adopts a double-branch convolution architecture and comprises symmetrical X coordinate and Y coordinate processing channels; according to the invention, through a double-branch convolutional neural network dynamic prediction spine algorithm and a biological-geometric constraint correction mechanism, the spine key point prediction precision and the physiological curvature reduction capability are significantly improved. Compared with a mainstream static geometric rule, the method has the advantages that the lumbar vertebra point offset under the pelvis rotating posture is reduced from 35 pixels to 5.6 pixels, the test set R2 reaches 99.77%, the data requirements of motion capture, medical treatment and the like are met, and the adaptation bottleneck of massive open source data is solved.

The application discloses a hotspot processing method and device in a database and a computer readable storage medium. The method comprises the following steps: when it is determined that a hotspot region exists in an HBase open source database, acquiring all primary key data corresponding to the hotspot region in the HBase open source database; processing all the primary key data by using a predetermined processing mode to process rowkey values, and obtaining new rowkey values; simulating access to the HBase open source database based on the new rowkey values; when it is determined that no hotspot region exists in the HBase open source database during the simulation access, determining that the predetermined processing mode is a target optimization mode; and optimizing the rowkey values by using the target optimization mode to process the hotspot region existing in the HBase open source database.