Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

49 results about "Security testing" patented technology

Security testing is a process intended to reveal flaws in the security mechanisms of an information system that protect data and maintain functionality as intended. Due to the logical limitations of security testing, passing security testing is not an indication that no flaws exist or that the system adequately satisfies the security requirements.

A multi-level quantitative evaluation method for an electronic product information clearing, verifying and tracing credible closed loop system

PendingCN122346411ATechnology developmentAttack
The application discloses a kind of multi-level quantitative evaluation methods for electronic product information clearing, verification and traceable credible closed-loop system, comprising S100, evaluation framework establishment and test environment preparation step: S110, define evaluation model, the evaluation model includes clearing effect layer (L1), verification credible layer (L2), trace and authentication layer (L3), system behavior and security layer (L4) and performance and efficiency layer (L5).The application has the advantages that: from qualitative to quantitative: a large number of quantitative indicators such as KL divergence, bit recovery rate, throughput are introduced, and the evaluation results are objective, accurate and comparable.The method carries out deep security testing: not only test function, but also take the system itself as attack target, carry out penetration testing and process behavior monitoring, can find deeply hidden design defects and security vulnerabilities, which cannot be achieved by traditional function testing.At the same time, the method can guide technology development, and provide decision basis for procurement and supervision;And high automation and repeatability.
Owner:GUIZHOU UNIV

A network protocol fuzzing method, device, storage medium and equipment

PendingCN122293562APathPingLinguistic model
This application provides a method, apparatus, storage medium, and device for network protocol fuzzing. The method introduces retrieval-enhanced generation technology into protocol fuzzing. By retrieving external knowledge, it provides context for a large language model, ensuring the accuracy of protocol specifications and thus improving the accuracy of generated test cases in protocol testing. Simultaneously, it utilizes the large language model to analyze the out-degree value of the state of the seed to be mutated, and then intelligently allocates test resources based on the out-degree value, thereby enhancing the efficiency of state space exploration. Thus, by combining the large language model with external knowledge retrieval capabilities, test cases that conform to protocol specifications and cover more protocol paths are generated, effectively improving the effectiveness and coverage of network protocol security testing.
Owner:BEIJING UNIV OF POSTS & TELECOMM

Python-based putty trusted software testing method and system

PendingCN122152699AError detection/correctionTest documentAutomatic control
The application belongs to the field of trusted software security testing, and discloses a putty trusted software testing method and system based on Python. The application replaces traditional manual operation with an automatic control mode taking Python as the core, so that the full test process of the SIS (Supervisory Information System) safety and trusted infrastructure can be quickly carried out after the version is released. The application automatically processes test case analysis, command execution, output collection, matching judgment, screenshot recording and report generation, thereby avoiding the cumbersome process of manually executing test instructions, manually comparing outputs, manually taking screenshots and manually arranging test documents.
Owner:HUANENG POWER INT CO LTD RIZHAO POWER PLANT +2

A large model threat sample library construction method

PendingCN122433830AEvaluation resultData mining
The application belongs to the technical field of large model security, and specifically discloses a large model threat sample library construction method, which comprises the following steps: step S1, embedding malicious instructions or malicious data into prompt words to construct threat samples; step S2, inputting the threat samples into a target large model as prompt words; step S3, using an evaluation large model to evaluate the output results of the target large model; and step S4, if the evaluation result is harmful content, adding the threat sample into a threat sample library, otherwise, discarding the threat sample. The application solves the problem of the lack of a threat sample library for security testing of an existing large model, and automatically tests and evaluates the threat samples during the construction of the sample library, so that the threat samples can be stored in the library only after the evaluation, thereby effectively improving the construction efficiency of the threat sample library. Meanwhile, the constructed threat sample library can support the security testing of the target large model, and can be used to improve the security of the target large model.
Owner:CHINESE PEOPLES LIBERATION ARMY UNIT 32802

A security test method, device, equipment, medium and product are applied

PendingCN122333454AApplication securityTesting Methods
The application provides an application security testing method and device, equipment, medium and product, which can be applied to the fields of artificial intelligence technology and financial technology. The method comprises the following steps: determining test information of a to-be-detected application in an operation test process; the test information comprises to-be-detected information called by the to-be-detected application in the operation test process of the to-be-detected application; determining a corresponding compliance baseline of the to-be-detected application based on a pre-trained baseline generation model; the compliance baseline comprises a compliance information call baseline; the compliance information call baseline is used for representing information allowed to be called by the to-be-detected application; determining a baseline deviation between the test information and the determined compliance baseline, and determining a security detection result of the to-be-detected application according to the baseline deviation; the baseline deviation comprises an information deviation between the to-be-detected information and the compliance information call baseline.
Owner:INDUSTRIAL AND COMMERCIAL BANK OF CHINA

Method and device for testing password security of cloud platform access of power monitoring system

This application relates to the field of cryptographic security testing technology, and discloses a method and apparatus for testing the access password security of a cloud platform in a power monitoring system. The method includes: first, acquiring access password records and role information to generate a password-role mapping table; then, constructing a permission coupling structure and adding a time weight to form a permission coupling evolution structure; collecting system evolution factor data and normalizing it into security erosion parameters; subsequently, starting with the access password, combining the permission coupling evolution structure and the set of evolution factors to construct a permission-aware security degradation model and calculate the path degradation amplification coefficient; based on this coefficient, generating a permission-time joint security test signal containing a comprehensive risk value, and determining the password security status by comparing it with a preset security threshold. This method can accurately identify high-risk passwords without affecting system operation, meeting the high security and high reliability requirements of the power industry.
Owner:CHINA ELECTRIC POWER RESEARCH INSTITUTE CO LTD

Method and system for automatically generating a network security testing mission profile based on subject matter

The application provides a method and system for automatically generating a network security test task outline based on subjects, comprising the following steps: S1, based on task information file decomposition index requirements, analyzing network security index keywords related to network security test in the index requirements; S2, based on the network security index keywords, using a related entity discovery technology to obtain recommended subjects corresponding to the index; S3, using a mind map to display the corresponding relationship between the measured system, the index and the subject; and S4, based on the corresponding relationship between the measured system, the index and the subject, using a network security test task outline template to automatically generate a test task outline.
Owner:NO 50 RES INST OF CHINA ELECTRONICS TECH GRP

A test case generation method for vulnerability version identification and related equipment

PendingCN122412304AAlgorithmSource code
This application discloses a test case generation method and related equipment for vulnerability version identification. This solution obtains a target function call sequence suitable for the target version by inputting the source code of the original version of the program under test, the function call stack when the vulnerability is triggered in the original version, the original test cases, and the source code of the original test cases. The target version of the program under test is run, and test cases that can trigger the vulnerability to be verified are generated through mutation. The function call status of the test process during execution is monitored, and the test process is terminated early if the function call status cannot satisfy the target function call sequence. The embodiments of this application use targeted fuzz testing technology to achieve high-efficiency vulnerability identification and verification with no false positives, and can generate test cases to facilitate vulnerability analysis by testers. It can be widely applied in the field of computer security testing technology.
Owner:GUANGZHOU UNIVERSITY

Large language model (LLM) supply chain security

PendingUS20260178728A1Platform integrity maintainanceDocumentationSecurity testing
Disclosed are various approaches for large language model (LLM) supply chain security. In one example, a system comprises a computing device that is configured to identify a large language model (LLM) application from a repository. A signed attestation document is added to the LLM application. The signed attestation document provides LLM specific supply chain information for the LLM application. The computing device executes an automated LLM security test of the LLM application and attaches, to the signed attestation document, a signed LLM security test attestation based at least in part on completion of the automated LLM security test.
Owner:AMERICAN EXPRESS TRAVEL RELATED SERVICES CO INC

A state protocol fuzzing method and process based on a hierarchical large language model framework

PendingCN122372472ALocal languageLinguistic model
This invention discloses a state protocol fuzzing method and process based on a hierarchical large language model framework, belonging to the field of cyberspace security technology. Addressing the problems of skill dilution, context obfuscation, and high computational cost inherent in existing single large language model-driven fuzzing methods, this invention constructs a syntax analysis layer, a semantic reasoning layer, and a policy optimization layer. Specifically, the syntax analysis layer extracts a structured syntax tree from the raw message using protocol templates and an LLM (Local Language Model); the semantic reasoning layer combines protocol state with the syntax tree to infer business logic constraints, identify vulnerability patterns, and dynamically assess risks; and the policy optimization layer intelligently generates and adaptively adjusts mutation strategies based on risk scores and historical feedback. This invention effectively improves the coverage depth of the protocol state space and the efficiency of vulnerability discovery, while reducing computational costs, providing an efficient, intelligent, and economical solution for the security testing of complex network protocols.
Owner:HANGZHOU INNOVATION RES INST OF BEIJING UNIV OF AERONAUTICS & ASTRONAUTICS +1

A voice security test method and device for smart home based on voiceprint anti-counterfeiting

PendingCN122455011AAttackHome based
The application discloses a kind of intelligent home voice security test method and device based on voiceprint anti-counterfeiting, it is related to intelligent home security test technical field.The core of the device includes main control module, attack simulation module and response acquisition module.Attack simulation module generates multi-dimensional voice deception attack sample including replay, synthesis, conversion and countermeasure sample.Main control module is built-in dynamic countermeasure reinforcement learning algorithm, and dynamically generates digital domain control instruction and physical domain control instruction according to the response state of the device to be measured, wherein the physical domain control instruction directly drives the stepper motor of physical replay sub-module to carry out angle rotation, realizes the deep binding of algorithm strategy and physical hardware action.Response acquisition module comprehensively captures the state feedback of the device to be measured by multi-modal perception means.The device can further include environment simulation module and evaluation and output module to construct physical test space with specific acoustic characteristics and output quantitative evaluation report.The application first proposes a comprehensive test device combining software and hardware from the perspective of active attack test, which can comprehensively and automatically evaluate the voiceprint anti-counterfeiting performance of intelligent home voice control system and Internet of Things devices.
Owner:ZHEJIANG SHENLING TECHNOLOGY CO LTD

An application installation method, electronic equipment, chip system and readable storage medium

This application provides an application installation method, an electronic device, a chip system, and a readable storage medium. The method includes: obtaining an installation package of a target application, target data, and a first signature corresponding to the target data. The first signature is obtained by signing the target data after the installation package passes security testing. The first signature is verified based on a target deployment certificate, which proves that a trusted object has the authority to deploy applications on devices managed by the trusted object. The target data includes an identifier of the device managed by the trusted object. If the first signature verification passes and a first terminal device belongs to a device managed by the trusted object, the target application is installed based on the installation package. Thus, the installation package installed on the first terminal device passes security testing, reducing the risk of installing the target application and improving the user experience.
Owner:HUAWEI TECH CO LTD

A method and device for generating fuzzing test data for vehicle networking security

InactiveCN122093170Aimprove pass rateReduce spawn rateSecuring communicationBus networksSimulationSecurity testing
This application relates to the field of security testing technology, specifically to a method and apparatus for generating fuzzy test data for vehicle-to-everything (V2X) security. The method includes: acquiring CAN bus data; calculating a parameter activity coefficient based on CAN bus frame data with the same ID in the real vehicle data, used to measure the activity and boundary conditions of various control commands in the real vehicle data; then combining the correlation between different IDs to perform temporal correlation feature measurement, and further calculating a random weight coefficient to measure the intensity of random transformation of each CAN bus frame data in the real vehicle data; and correcting the standard deviation of Gaussian noise based on the activity of different types of data and the dependencies between data, thereby generating test data for V2X security fuzzy testing. This application aims to generate test data that conforms to the statistical laws of vehicle communication, thereby improving the data pass rate in the underlying protocol stack and thus more effectively testing the upper-layer business logic.
Owner:NAT IND INFORMATION SECURITY DEV RES CENT

Agent-driven semantic clustering directed fuzzing method and apparatus

The embodiment of the application discloses an agent-driven semantic clustering directed fuzzing method and device, and relates to the technical field of software security testing.The method specifically comprises the following steps: firstly, a target point is acquired and is merged according to functions; secondly, the control flow graph of each function is analyzed before testing, key predicates are extracted, and a constraint signature is generated for the target point; the constraint signature is divided into semantic clusters based on a logical relationship, and combined constraint signatures of the semantic clusters are formed in combination with calling contexts; then, in the testing runtime, the semantic clusters are taken as units, seeds and energy are dispatched according to the combined constraint signatures of the semantic clusters, the state is updated according to feedback, and it is determined whether the semantic clusters are stagnant; when the semantic clusters are stagnant, the input is directed to be adjusted to generate a new seed and to be executed based on the combined constraint signatures of the semantic clusters and the path of the current seed; finally, the state of the semantic clusters and the historical strategy are updated according to the execution result of the new seed, and the above-mentioned dispatching and generating process is iteratively executed, so that efficient and accurate vulnerability testing is realized.
Owner:XIAMEN UNIV OF TECH

Binary malware adversarial sample generation method and system based on large language model

PendingCN122286762AOverlayLinguistic model
This invention discloses a method and system for generating adversarial samples of binary malware based on a large language model, belonging to the field of cybersecurity adversarial testing technology. First, the input PE file sample undergoes structural perturbation and verification, and its multidimensional static features are extracted. Based on these features, a context-aware jailbreaking strategy is introduced, and a contextual prompt template for circumventing the content security strategy of the large language model is constructed. This template drives the large language model to generate adversarial overlay code, which is then expanded according to a predefined strategy and appended to the end of the structurally perturbated PE file sample to form a preliminary adversarial sample. The preliminary adversarial sample is then input into a malware detector for testing. Based on the test results, the strategy is dynamically adjusted, and the final adversarial sample for security testing is output. This invention systematically verifies the dual value of the large language model in binary adversarial generation for the first time, constructs a scalable attack and defense testing framework, and provides theoretical support for a new generation of detection systems.
Owner:ZHEJIANG UNIV

A software security vulnerability intelligent scanning method based on static analysis

PendingCN122331950AProgram graphVulnerability
This invention discloses an intelligent scanning method for software security vulnerabilities based on static analysis, specifically relating to the field of software security testing technology. The method acquires a snapshot of the baseline version source code and a snapshot of the modified version source code corresponding to the software project under test, establishes alignment relationships between program elements of the previous and current versions, and forms a set of modified elements. Combining the baseline version program graph cache, function summary cache, and historical hazard index, a comprehensive quantity of the impact of changes is formed, and a subgraph of the impact of changes is constructed. Static incremental vulnerability analysis is performed on the subgraph of the impact of changes to identify newly added risk paths and activated historical hazard paths, and a trusted comprehensive quantity of risk activation is formed. Based on the trusted comprehensive quantity of risk activation, confirmation screening and baseline updates are performed, and valid incremental alerts or paths awaiting review are output. This invention can narrow the analysis scope, reduce the resource consumption of full scans, and improve the ability to identify incremental vulnerabilities and the activation of historical hazard paths.
Owner:SHANGHAI XUYIN TECHNOLOGY CO LTD

Method, device, equipment, medium and product for safety detection of analog telephone line

This application discloses a method, apparatus, device, medium, and product for security testing of analog telephone lines. The method includes: acquiring the voltage and current of the analog telephone line to determine its line status; collecting the dual-tone multi-frequency (DTMF) signal, voice signal, and line impedance of the analog telephone line, as well as acquiring historical behavior data of the analog telephone line; determining whether the DTMF signal, voice signal, line impedance, and historical behavior data meet predetermined security constraints to obtain a condition judgment result; and inputting the condition judgment result into a pre-constructed weighted risk scoring model for security assessment to determine the security testing result of the analog telephone line. According to the embodiments of this application, the false alarm rate of analog telephone line security testing is effectively reduced, detection sensitivity and scalability are improved, and public communication security and operator reputation are effectively protected.
Owner:CHINA MOBILE COMM GRP TERMINAL +1

An automated safety test scheme generation method and system for intelligent networked vehicles and a medium

PendingCN122284579ATest scriptAttack
This invention relates to a method, system, and medium for generating automated security testing schemes for intelligent connected vehicles. The method includes: constructing an attack tactics knowledge graph; collecting and parsing asset configuration information and network topology information of the target vehicle system; generating attack strategies based on a multi-agent collaborative mechanism; establishing a semantic mapping library between attack techniques and attack tools, and generating tool execution parameter configurations according to the configuration parameters of the target vehicle system; generating a structured attack test scheme document and converting it into an executable attack test script sequence; executing the attack test script sequence, capturing test result data during execution, and updating the confidence scores of attack technique nodes and the effectiveness scores of attack tool nodes in the attack tactics knowledge graph based on the test result data. This invention improves the automation, intelligence, and executability of generating attack test schemes for intelligent connected vehicles.
Owner:HARBIN INSTITUTE OF TECHNOLOGY (SHENZHEN) (INSTITUTE OF SCIENCE AND TECHNOLOGY INNOVATION HARBIN INSTITUTE OF TECHNOLOGY SHENZHEN)

An in-vehicle network protocol fuzzing method, platform, device and storage medium

This invention discloses a fuzz testing method, platform, device, and storage medium for vehicular network protocols. These are corresponding solutions. The solutions utilize a large language model to automatically parse protocol semantics, enabling intelligent generation of test cases, reducing reliance on manual protocol analysis, and improving testing efficiency and coverage. Furthermore, semantically aware mutation strategies enhance test case quality and improve the ability to discover deep protocol logic vulnerabilities. In addition, protocol parsing, test case generation, test execution, and result analysis are integrated into a single design, achieving automated and closed-loop management of the testing process, thus improving the overall performance and practicality of vehicular network protocol security testing.
Owner:UNIV OF SCI & TECH OF CHINA

Application behavior security testing methods, devices, equipment, media and program products

PendingCN122134445Aprevent theftPrevent tamperingFinanceDigital data protectionRisk profilingTerminal equipment
This application provides a method, apparatus, device, medium, and program product for security detection of application behavior, relating to the financial technology field. It includes: based on the startup status of a banking application installed on a terminal device under test, in a trusted execution environment corresponding to the security chip of the terminal device, real-time collection of sensitive behavior data of the banking application; in the trusted execution environment, risk analysis processing of the sensitive behavior data is performed based on the historical usage data of the banking application corresponding to the terminal device to determine the risk category of the sensitive behavior data; based on the risk category, a security report for the banking application is generated, and a security report generation prompt is displayed; in response to the user's viewing operation of the prompt, abnormal behavior verification is performed on the terminal device and the banking application in the trusted execution environment; if the verification is successful, the security report is de-identified to obtain and display a de-identified security report. The method of this application improves the security of application behavior detection.
Owner:INDUSTRIAL AND COMMERCIAL BANK OF CHINA

Option-aware directed grey-box fuzzing vulnerability concept verification automatic generation method

PendingCN122285520Aprecise prior knowledgeAvoid invalid mutationsGrey boxTest phase
This invention belongs to the field of software security testing technology, specifically a method for automatically generating vulnerability proof-of-concept (PoC) based on option-aware directed gray-box fuzzing. The invention includes: performing static analysis on the target program, extracting each configuration option and its directly affecting variables, analyzing the complete value space and control conditions of the variables, and inferring the constraint relationships between options; based on option knowledge, employing option validity inference technology based on taint analysis during the fuzzing phase to dynamically identify option combinations effective in reaching the target location; introducing an alternating guided fuzzing strategy, alternately executing option input fuzzing and file input fuzzing, systematically driving driver execution to approximate the target vulnerability location, and ultimately efficiently generating the target vulnerability PoC. This invention effectively solves the problems of traditional directed gray-box fuzzing tools ignoring the influence of program configuration options, the independence of options and file mutations, and the lack of coordination, significantly improving the efficiency and success rate of vulnerability PoC generation.
Owner:FUDAN UNIVERSITY

Static application security testing tool comprehensive evaluation method, device and equipment

PendingCN122450794ASoftware quality assurancePerformance index
The application belongs to the technical field of software quality assurance and security testing, and specifically discloses a static application security testing tool comprehensive evaluation method, device and equipment. Through the application, a configuration template library is called to load preset detection configurations for each static application security testing tool; each static application security testing tool after loading the configurations is controlled to execute an evaluation task; a comprehensive quantitative score is calculated according to multi-dimensional index values, and the corresponding static application security testing tool is comprehensively evaluated according to the comprehensive quantitative score. In the above manner, the evaluation indexes are defined in the vertical dimension of 'three verticals', the detection capability index, the performance index and the availability index are included in the same framework for systematic and quantitative evaluation, a stereoscopic decision basis is provided for tool selection, and then the corresponding static application security testing tool is comprehensively evaluated according to the comprehensive quantitative score, so that the comprehensiveness and accuracy of the evaluation tool can be effectively improved.
Owner:CHINA STATE SHIPBUILDING CORP LTD RESEARCH INSTITUTE 719 +1

An attack scenario script variation method, device and medium

This application belongs to the fields of network security testing and network range technology, and discloses a method, device, and medium for mutation of attack simulation scripts. This application enables the server to strengthen the direction of potential risks based on the current attack simulation script and the built-in expert knowledge base. By mutating the direction of potential risks, the anomaly coverage of potential risks is improved. At the same time, a random perturbation strategy is used to ensure that as many potential risks as possible can be discovered in multiple rounds of mutation iterations. After a risk is discovered, the mutated script is merged into the system's built-in script library, thereby achieving the goal of automatically expanding the script library and reducing the input of manually generated scripts.
Owner:NO 30 INST OF CHINA ELECTRONIC TECH GRP CORP

Method for testing security of baseboard management processor and electronic device

PendingCN122450757APathPingData set
The application discloses a security test method of a baseboard management processor and electronic equipment, relates to the technical field of testing, and comprises the following steps: performing multi-domain hierarchical mapping on a baseboard management controller security scheme, generating a hierarchical test object set of a security domain, and extracting test objects corresponding to each domain security scheme; constructing an attack path across at least two types of test object sets and instantiating the attack path into a scene sample, reproducing remote firmware upgrade and access control linkage complex working conditions; carrying out multi-dimensional cross-domain testing containing firmware upgrade and access control interaction based on the scene sample, collecting interactive test data sets containing execution data flow, actual response and expected response, and completely retaining multi-module linkage test data; performing feature analysis and result aggregation on the data set to generate a verification output set, so that the protection capability is systematically verified, the overall protection effect in a complex scene is accurately evaluated, and the defects that related technologies lack systematic testing means, protection capability verification and accurate evaluation of protection effect in a complex scene are solved.
Owner:INSPUR SUZHOU INTELLIGENT TECH CO LTD

Security testing method and device for model and storage medium

This application discloses a security testing method, apparatus, and storage medium for a model. Relating to the field of artificial intelligence, the method includes: obtaining a set of test questions, wherein the set of test questions includes at least two of the following: a first type of question, a second type of question, and a third type of question; the first type of question is a preset question; the second type of question is generated based on entity combination and a question template; and the third type of question is generated by a question generation model based on the first type of question and / or the second type of question; processing the set of test questions using a target question-answering model to obtain a set of answer content corresponding to the set of test questions; and determining the security test result of the target question-answering model based on the set of answer content. This application solves the problem that related technologies rely on manually written attack test samples for security testing of question-answering models, resulting in low accuracy of security test results.
Owner:BEIJING CALORIE INFORMATION TECH CO LTD

Cross-team software development collaboration optimization system and method

This invention discloses a cross-team software development collaborative optimization system and method, relating to the field of data collaborative optimization. The invention collects software requirement data from information sources and language libraries from multiple software development teams, processes this data, and then constructs language library collaborative awareness rules and security testing rules through rule definition. Based on these rules, a development collaboration model is built. Simultaneously, the development collaboration processes of each collaborator in multiple software development teams are collected, and the collaborative development processes of each collaborator are synchronized based on the constructed development collaboration model, the processed software requirement data, and a wireless network. Finally, the collaborative development processes of each collaborator are verified and optimized based on the processed information source software requirement data and security testing rules, thereby improving the effectiveness of software development collaborative optimization.
Owner:BEIJING CENTURY YUANXIANG TECH CO LTD

Method, device and equipment for testing protocol flood attack function and storage medium

PendingCN122293440ATest performanceAttack
This application discloses a testing method, apparatus, device, and storage medium for Internet Control Message Protocol (ICP) flood attack functionality, relating to the field of network security testing technology. The method includes: acquiring test parameters and generating test cases based on the test parameters; collecting baseline performance data of a target device; sending a traffic attack generated based on the test cases to the target device and collecting test performance data of the target device during the traffic attack; determining protection strategy optimization parameters based on the test performance data; writing the protection strategy optimization parameters into the target device for secondary testing and collecting optimized performance data of the target device during the secondary test; and comparing the optimized performance data with the baseline performance data to verify the optimization effect. This application improves the iteration efficiency and verification reliability of Internet Control Message Protocol (ICP) flood attack protection strategies.
Owner:SHENZHEN FENGRUNDA TECH CO LTD

Open source large language model automatic alignment test method based on multi-strategy fusion

PendingCN122365509AData setAlgorithm
This application relates to the field of large language model security testing technology, proposing an automated alignment testing method for open-source large language models that integrates multiple strategies. Based on text interpretation preprocessing and adaptive parameter adjustment, this method is geared towards secure alignment evaluation of open-source large language models. It introduces a structured multi-point crossover mechanism, a dynamic hybrid mutation mechanism, and an intelligent optimization termination module into the evolutionary search process. Experiments on the AdvBench dataset show that this method achieves attack success rates of 0.9942, 0.9923, and 0.8577 on Vicuna-7B, Guanaco-7B, and Llama2-7B-Chat, respectively, with an average runtime of 167.95 seconds, an average of 13.288 GPT calls, and a single-sample cost of $0.0021, significantly improving the efficiency and effectiveness of automated alignment testing.
Owner:XIAN UNIV OF POSTS & TELECOMM