53 results about "Duplicate detection" patented technology

The invention relates to the technical field of code analysis, and discloses a multi-terminal code repetition detection and component reconstruction method, device, equipment and medium, and the method comprises the steps: carrying out code analysis on a pre-acquired project code file, and identifying a code mapping relation to obtain an abstract syntax tree and a cross-platform code mapping relation, and according to the abstract syntax tree and the cross-platform code mapping relationship, carrying out repetition logic detection on the project code file to obtain a repetition detection result, according to the repetition detection result, confirming a repetition code in the project code file, carrying out cross-platform code optimization on the repetition code to obtain an optimized code, and carrying out cross-platform code optimization on the optimized code. The method comprises the steps of obtaining duplicated codes, extracting reusable components on the basis of the duplicated codes, generating a multi-end adaptive component template according to a preset condition compiling logic, and reconstructing the optimized codes according to the reusable components and the multi-end adaptive component template to obtain reconstructed codes, so that the code duplicating detection and reconstruction efficiency is improved.

The invention discloses a cloud password service-oriented fuzzy test and compliance evaluation platform and method, and relates to the technical field of cloud computing and password security evaluation. The evaluation platform comprises a demand analysis module, a test case generation module, a fuzzy test execution module, a compliance detection module, a result fusion analysis module and a report generation module, and the result fusion analysis module is connected with the fuzzy test execution module and the compliance detection module. And the cloud password service detection module is used for carrying out association analysis on the response data, the abnormal log and the compliance detection result acquired by the fuzzy test, identifying the security vulnerability type and the compliance defect level of the cloud password service, and removing a repeated detection result. Fuzzy testing and compliance detection are organically fused, correlation analysis of the two results is achieved through the result fusion analysis module, the problems that in the prior art, the two tests are separated, and the results are difficult to integrate are solved, and the safety compliance condition of the cloud password service can be comprehensively reflected.

The embodiment of the invention provides an online man-machine conversation optimization method, system and device, electronic equipment, a storage medium and a program product. In the scheme provided by the embodiment, when a first dialogue generation model executes a streaming response based on user input data, a currently generated first dialogue text fragment is subjected to repeated detection, and when it is detected that repeated content exists in the first dialogue text fragment, the first dialogue generation model is controlled to interrupt execution of the streaming response; in addition, reply generation constraint information is determined according to the repeated content in the first reply text segment, and then reply generation cue words are generated based on the reply generation constraint information and the non-repeated content in the first reply text segment. And after the dialogue generation cue word is input into the first dialogue generation model, the first dialogue generation model is triggered to determine a dialogue generation starting point according to the non-repeated content in the first dialogue text fragment, and streaming response is continuously executed from the dialogue generation starting point.

The invention discloses an approximate repetition detection method and system fusing local retrieval and multi-dimensional decision. The method comprises the steps that a to-be-detected document is preprocessed, and text content and service meta-information are extracted; utilizing MinHash to generate a compact signature; quickly recalling a candidate set from a massive historical library through an LSH index; adopting a paragraph-level weighted Jaccard algorithm to accurately calculate the structural similarity between the candidate document and the document to be detected; when the similarity exceeds a threshold value, intelligent decision making is carried out according to a preset priority chain considering multi-dimensional business rules such as release mechanism levels and time, and finally reserved documents are determined; and recording the structured audit log in the whole process for tracing. The method realizes approximate repetition detection with high recall, high precision, low time consumption and interpretable and auditing decision process, and is especially suitable for massive de-duplication scenes of strong normative texts such as government affair official documents, news announcements and the like.

The application discloses a multi-source data fusion method and device, electronic equipment and storage medium. The method comprises the following steps: acquiring source data in at least one source database and mother database data in a data mother database; performing quality evaluation on the source data and the mother database data to obtain a quality evaluation result; for any source data, performing a duplicate detection process on the source data and the mother database data to obtain a duplicate detection result; and performing data fusion on the source data and the mother database data based on the duplicate detection result and the quality evaluation result. The application increases the consideration of the quality of source data and mother database data, avoids the influence of low-quality data on fused data, and improves the data quality of the mother database data after fusion.

Disclosed are systems and methods including software processes executed by a server that detect audio-based synthetic speech (“deepfakes”) in a call conversation. The server applies an NLP engine to transcribe call audio and analyze the text for anomalous patterns to detect synthetic speech. Additionally or alternatively, the server executes a voice “liveness” detection system for detecting machine speech, such as synthetic speech or replayed speech. The system performs phrase repetition detection, background change detection, and passive voice liveness detection in call audio signals to detect liveness of a speech utterance. An automated model update module allows the liveness detection model to adapt to new types of presentation attacks, based on the human provided feedback.

The invention discloses a weak event space-time calibration and metastable state heat perception method and system for an emergency scene. The method comprises the following steps that: a client acquires an event type, an event timestamp, a position coordinate and media information and uploads the information to a server; the server performs time and space dimension repeated detection on the events, and writes the de-duplicated events into a buffer area; calculating a time offset based on the client event time and the server receiving time, and carrying out unified calibration on the event time; re-estimating the confidence coefficient of the event position in combination with the regional positioning quality; counting an event position offset trend in a preset time window, identifying the overall positioning drift of the region, and performing position correction when the overall positioning drift exceeds a threshold value; self-adaptively constructing a metastable state time window for weak event aggregation according to the change of the number of events in the window; and based on the corrected position, the revaluation confidence and the distance attenuation weight, performing weighted stacking on the events in the window to generate a dynamic popularity value.

The invention provides a bidding document duplicate checking method, system and device and a medium. The method comprises the steps of obtaining file content of a bidding document; the bidding document comprises a document to be subjected to duplicate checking and a comparison document; extracting sub-strings of the file content; performing bucket dividing processing on the sub-character strings, and mapping the sub-character strings meeting a preset condition into the same hash bucket; calculating the similarity between the sub-character strings of the document to be subjected to duplicate checking and the sub-character strings of the comparison document in the same hash bucket, and determining the pairing relationship between the sub-character strings; performing pairing and merging according to the pairing relationship to obtain a maximum repeated fragment; and calculating a content repetition rate according to the maximum repetition fragment so as to determine a duplicate checking result of the bidding document. Similar sub-strings are concentrated in the same hash bucket through a bucket dividing strategy, so that rapid repeated detection of large-scale text data is realized, and the duplicate checking response speed is increased; and determining the maximum duplicate fragment according to the similarity and the pairing relationship, thereby solving the problem that duplicate fragments with a small number of altered characters cannot be identified.

This system more accurately detects duplicate evidence and prevents the inclusion of duplicate documents. [Solution] The duplicate detection unit P5 detects a duplicate between the source document information Ss and the referenced document information Sd when confirmed information F included in the source document information Ss is input, and the confirmed information F included in the source document information Ss is common with at least one of the confirmed information F or estimated information U included in the referenced document information Sd.

The present application relates to the technical field of railway detection, and discloses a railway line video equipment extraction method and device based on unified identification and time sequence fusion of multiple types of equipment. It aims to solve the technical problems of lack of unified identification ability of multiple types of key equipment, high false detection, serious repeated detection, low detection efficiency and insufficient robustness caused by single frame detection in the prior art. The present application includes the following steps: video acquisition and image preprocessing; generation of equipment candidate area based on prior knowledge of the railway industry; unified identification of multiple types of key equipment; construction and association of time sequence trajectories of continuous frame detection results; equipment existence confirmation based on time sequence fusion; equipment level result output and spatial deduplication. From the complete processing link of vehicle-mounted video data to equipment level structured results. The present application realizes unified identification and extraction of multiple types of railway 2C key equipment, significantly reduces false detection and repeated detection by using time sequence fusion, and improves detection efficiency and robustness by introducing prior knowledge of the railway industry.

The embodiments of the present application relate to the field of big data. Provided are a method and apparatus for determining duplicate data, and a device. The method comprises: on the basis of a merged computing node, acquiring bitmap arrays and single-machine duplicate sets of M computing nodes; performing pairwise bitwise operation processing on the bitmap arrays of the M computing nodes, so as to obtain M duplicate values; acquiring local reverse dictionaries corresponding to the computing nodes, and on the basis of the local reverse dictionaries, performing secondary duplicate check processing on the duplicate values, so as to obtain multi-machine duplicate sets corresponding to the duplicate values; and merging the multi-machine duplicate sets and the single-machine duplicate sets, so as to obtain a global duplicate set. The method in the present application improves data duplicate detection efficiency.

The application discloses a kind of graph node mixed traversal sorting method and system of workflow visualization, it is related to data visualization technical field.The method includes: obtaining workflow graph and initializing data structure;From start node, perform depth-first mixed traversal, repeat detection and block to node in the traversal process;After traversal is completed, the coordinates of each node are calculated, and the secondary sorting is executed to the node list, finally, continuous serial number is distributed and the sorting result is output.The application realizes zero repeat processing by throttling blocking mechanism, realizes zero data backflow by branch anti-backflow control mechanism, improves the sorting accuracy by secondary sorting algorithm, has excellent real-time response performance, and can be widely applied in workflow visualization, ETL task arrangement, low-code development platform and other fields.

A system filters near-duplicate images to generate data for training or validation of a machine learning model. The system receives a set of images and generates feature vectors from the images. The system clusters the feature vectors. For each cluster of feature vectors, the system determines near-duplicate pairs of images. The system may generate a cost matrix representing a linear assignment problem and find near-duplicate pairs of images by solving the linear assignment problem. The system filters images from the set of images based on the near-duplicate pairs of images. The system uses the filtered set of images for training or validation of the machine learning model.

Proposed is a digital system using Locality Sensitive Hashing (LSH) for near duplicate detection on weighted datasets providing a robust technical solution to the problem by improving (i) efficiency by allowing for fast and accurate identification of near duplicates in large datasets; (ii) accuracy by extending the hashes calculated via 5 the LSH approach with an insurance-specific weight, thus making sure that the concept of similarity between documents is calculated on a technical and industry-relevant basis; (iii) cost reduction by reducing redundant data lowers storage and processing costs; (iv) data integrity by improved accuracy and consistency in records enhance overall data integrity; and (v) regulatory compliance matching by automated data 10 management practices supporting compliance with industry regulations.

The present application provides a kind of dense translucent fry detection method, system and storage medium, it is related to computer vision technical field, the present application is by the transparency of target quantification, and as the basis, in feature fusion stage to weak signal is compensated and enhanced, while in model training stage to loss function is self-adaptive adjustment, to systematically solve the bottleneck problem such as missing detection, recheck and inaccurate positioning that existing technology faces when processing dense, small, translucent target.The detection recall rate of low contrast, translucent target is improved, the missing detection is effectively reduced;The detection accuracy and robustness in high-density, overlapping scene are improved, and the repeated detection is reduced;The positioning accuracy of model to edge fuzzy target is enhanced;By introducing physical priori knowledge, the overall generalization ability of model and applicability in complex underwater environment are improved.

Systems and methods of the present disclosure enable a processor to automatically detect duplicate data entries by receiving data entries associated with a user, where each data entry includes a value, a time, an entity identifier, and a location. Pairs of similar data entries are determined by matching the entity identifier and the location pairs data entries. Candidate duplicate data entries are determined based on a proximity in time between data entries of the similar data entries. For each candidate duplicate data entry, a feature vector is generated including the entity identifier, location, value and time, and each feature vector is submitted to a duplicate classification model to automatically determine duplicate data entries from the candidate duplicate data entries, the duplicate classification model being trained according to a historical dispute entries.

The invention discloses an anti-counterfeiting product production monitoring and duplicate checking detection method, system and device and a medium, and the method comprises a unique identification generation method which is used for converting product number batch and page number information for identifying the uniqueness of a product into a bar code and drawing the bar code to a printing stock bleeding position; the production monitoring and duplicate checking detection and online comparison method is used for carrying out production whole-process monitoring, duplicate checking detection and online comparison on the unique identification information. According to the technical scheme, the whole production process of the anti-fake product is monitored, the production progress is monitored in real time, duplicate checking detection is achieved for the anti-fake product with one object and one code, and repeated codes are completely eradicated.

The application discloses a kind of based on the semantic focus of test question duplicate checking method of large language model, the method is based on corpus construction and text normalization, complete the construction of question corpus and metadata annotation;Using semantic vectorization representation strategy, the question in corpus is mapped to dense vector, and offline semantic vector library is constructed;Propose semantic vector recall→SimHash denoising→Reranker model rearrangement screening mechanism, solve the problem that traditional literal comparison cannot identify synonymous rewriting;Propose multi-level screening-large model deep judgment-online threshold self-adaptive cooperation architecture, through real-time feedback continuous iteration, realize semantic level accurate duplicate checking;By starting mechanism, according to the data in artificial review library Dynamic adjustment vector recall threshold and large model deep judgment threshold, to maximize the accuracy and efficiency of duplicate checking.The application solves the problem of synonymous rewriting, short text representation failure, static threshold false alarm / miss.

The invention provides a weighted hash text duplicate checking method based on semantic weight and template suppression. The method sequentially comprises the following steps: extracting lexical, syntactic and semantic features; calculating word / segment / structure three-level template scores in an online stage based on an offline constructed template dictionary, and carrying out weighted fusion on the word / segment / structure three-level template scores; outputting the feature importance degree by the semantic weighting network subjected to comparative learning training, and combining the feature importance degree with the smooth IDF and the template inhibition factor to form a final weight; according to the weight, generating a weighted SimHash fingerprint or a weighted MinHash fingerprint; the fingerprints are written into a GPU accelerated multi-table LSH index and retrieved, the Hamming distance of the bit fingerprints is calculated, an adaptive threshold value is adopted for judgment, and when and only when the distance is not larger than the threshold value, a near-repetition detection result is output, and a document identifier, the distance and the threshold value are recorded. According to the method, virtual high similarity caused by template content is effectively inhibited, the detection rate and accuracy of anti-plagiarism such as word rewriting are improved, and the method has high efficiency and expandability.

The invention discloses a project combined type assembly editing method, device and equipment, a medium and a program product, and relates to the technical field of intelligent manufacturing. The method comprises the following steps: when a sub-project import operation is detected in a main project editing page, obtaining a current sub-project assembly file; according to the current sub-project identifier, carrying out primary repeated detection on the current sub-project assembly file; when the primary repeated detection of the current sub-project assembly file is passed, copying the current sub-project assembly file to obtain a current sub-project assembly copy; and splitting the current sub-project assembly copy to obtain a current sub-project cabinet three-dimensional diagram, a current sub-project detail table, a current sub-project terminal table, a current sub-project wiring table, a current sub-project cable table and a current sub-project wiring harness table, and performing hierarchical display in the main project editing page. According to the technical scheme provided by the embodiment of the invention, the editing efficiency and hierarchical display effect of project combined assembly are improved.

The invention discloses a 3D printing process self-adaptive regulation and control method based on machine learning, and relates to the technical field of additive manufacturing, and the method comprises the following specific steps: extracting genome DNA or cfDNA from a sample, and storing for later use; then designing a specific primer and a probe aiming at the IGVDJ conserved sequence; carrying out primer verification, and carrying out sequencing verification after electrophoresis detection; after the primer verification is passed, digital PCR detection is carried out to ensure that the contrast meets the requirement; finally, increasing the loading amount of the reinspection sample, forming a repeated hole, repeatedly detecting and summarizing data; according to the method, a multi-modal data synchronous acquisition and space-time cross fusion system is constructed, multiple types of sensors are integrated to obtain comprehensive signals, a fusion feature flow is generated through an algorithm, and the defect judgment reliability is improved in combination with double-branch deep learning; and a material defect cooperative regulation and control and double-model iteration self-updating mechanism is established, a parameter adaptation scene is dynamically adjusted, closed-loop optimization is formed, 3D printing stability and adaptability are enhanced, and intelligent and high-precision development of the technology is promoted.

The application discloses a path equipment state detection method and device, relates to the technical field of computers, and comprises the following steps: at least one volume mapped to a host is scanned, and at least one path equipment is determined according to the at least one volume; historical detection data in a preset period is collected through a sliding window, and a current detection interval length corresponding to different equipment states is determined in combination with equipment states, the number of path equipments and a proportion factor; according to the current detection interval length, state detection is performed on path equipments in different equipment states through a plurality of detection threads, a user state detection result of the at least one path equipment is determined, and in response to the fact that an abnormal component type of the path equipment is a link component, a target path equipment corresponding to the link component is determined, and the equipment state is synchronized to an abnormal state, so that the problems of repeated detection and untimely detection of a same-source fault are solved, the interval time of each round of state detection is dynamically adjusted, and the fault judgment logic is simplified.

This invention provides a method for constructing high-quality metal material process datasets based on a large language model, relating to the field of process data collection and processing technology. It introduces a chemical composition hashing (HC) mechanism, which transforms complex composition information into unique identifiers through standardized concatenation of a preset element set and MD5 hashing, enabling efficient and accurate cross-file comparison. A three-level duplicate detection system—precise duplicates → similar duplicates → LLM intelligent confirmation—effectively avoids false rejections due to differences in expression while ensuring high recall. A prompting engineering system is constructed based on a locally deployed large language model, integrating domain knowledge templates and strict output format constraints. This achieves complete reconstruction and structured expression of complex process chains without manual intervention, enabling end-to-end extraction of standardized material entries from massive amounts of unstructured documents. Furthermore, semantic-level feature hashing and LLM-assisted judgment mechanisms significantly improve deduplication accuracy and logical consistency.

The invention discloses a motor twin trawling system coupling centering calibration method, and belongs to the technical field of coupling centering calibration, and the method comprises the following steps: collecting the readings of axial and radial measurement meters at the 0-degree, 90-degree, 180-degree and 270-degree rotation positions of a coupling, and obtaining the diameter of the coupling, the distance of motor support legs and other geometric parameters; inputting the data into a calculation device, and automatically calculating an axial opening deviation and a radial deviation deviation based on a preset mathematical model; according to the deviation value and the geometric parameter, automatically solving the adjustment amount of the front and rear support legs of the motor in the vertical and horizontal directions through an adjustment amount algorithm model; generating and outputting visual guidance information containing the adjustment amount; the operator adjusts the motor supporting legs according to the guidance information, and detection is repeated until the centering precision meets the requirement; according to the invention, full-automatic calculation and real-time feedback are realized, the calibration precision and efficiency are effectively improved, personal errors are reduced, and the method is suitable for high-precision centering calibration of the coupler of the motor twin trawling system.

In one aspect, a method includes associating an MLO device with 2 transmission radios to yield a first communication link between the device and a first MLO access point and a second communication link between the device and a second MLO access point, wherein the device is configured to label the first communication link as a primary link and the second communication link as a secondary link; generating a sequence number to be assigned to a frame to be transmitted on the primary link and to a duplicate copy of the frame to be transmitted on the secondary link; associating a flag with the duplicate copy of the frame on the secondary link; and sending, from the device to the first and the second MLO access points, the frame and the duplicate copy on the primary link and the secondary link, respectively.

This invention discloses a method, system, and medium for shortwave time-frequency map target detection based on multi-map patch fusion, belonging to the field of shortwave communication technology. The method includes the following steps: time-frequency map construction; dividing the time-frequency map and constructing a multi-channel joint detection input containing a central patch and at least one adjacent patch; performing feature extraction and context fusion processing to obtain a fused feature map; performing target detection processing on the fused feature map and outputting candidate detection results for each patch; performing cross-map weighted non-maximum suppression processing, fusing overlapping candidate boxes across patches, and outputting a weighted fused candidate box set; mapping the weighted fused candidate box set to the original time-frequency map coordinate system to obtain the final target detection result. The method in this invention effectively avoids the loss of contextual information caused by processing the time-frequency map by dividing it into multiple patches, ensuring the integrity of targets in the edge regions of each patch, and avoiding missed detection of weak signals and repeated detection across patches.

The invention provides a security detection method, electronic equipment, a storage medium and a program product, and relates to the technical field of communication. According to the method, the credibility of the network data packet source equipment is firstly judged, and then the unpassed target security detection items are selectively executed based on the state indication information of the preset position, so that repeated detection of the data packets among trusted equipment in a trusted domain is avoided, the calculation load of the network equipment is greatly reduced, the data forwarding delay is effectively reduced, and the network security detection efficiency is improved. The transmission efficiency of the whole network is improved; and meanwhile, targeted detection is only carried out on items which do not pass the detection, so that the detection efficiency is considered on the premise of ensuring the network security protection strength.

The application provides a post-processing method of a target detection algorithm and an electronic device, and the specific steps comprise: obtaining a plurality of detection boxes of target recognition, each detection box being associated with a confidence score, category information and a granularity level, the confidence score being used to represent the probability of the corresponding detection box containing a target to be recognized, the category information being used to indicate the category of the target to be recognized in the corresponding detection box, and the granularity level being used to indicate the degree of granularity of the category indicated in the category information; determining the category granularity weight corresponding to each detection box according to the granularity level of each detection box; calculating the comprehensive priority score corresponding to the detection box by weighting processing through the confidence score of each detection box and the corresponding category granularity weight; and performing non-maximum suppression processing on the plurality of detection boxes to retain the detection box with the highest score and suppress the detection box overlapping with the retained detection box by more than a threshold. The above method avoids repeated detection caused by multi-granularity categories.