Model detection method, device, equipment and storage medium

Abstract

The present disclosure provides a model detection method and device, equipment and storage medium, relating to the field of artificial intelligence. The method comprises: obtaining corpus information, preprocessing the corpus information to obtain preprocessed corpus information, processing the preprocessed corpus information based on at least one of keyword extraction, clue generation and template generation to obtain test data. The model to be tested is detected based on the test data. Through the construction of test data for model detection, the detection of the model can be more accurate.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence, and in particular to a model detection method, apparatus, device, and storage medium. Background Technology

[0002] With the rapid proliferation of large language models, their security protection has become increasingly prominent. Existing large language models are vulnerable to various forms of attacks, including inducing the model to execute malicious queries through prompt injection, or bypassing the model's security restrictions using adversarial perturbations to generate illegal or dangerous content. These adversarial attacks have strong cross-model transferability and can even be automatically generated and spread rapidly across networks, causing serious security risks. Furthermore, the generation and processing of adversarial perturbations often involve high computational complexity. Applying protection strategies directly to large models not only increases computational overhead but also introduces efficiency bottlenecks, further increasing the difficulty of protecting large models. Summary of the Invention

[0003] This disclosure provides a model detection method, apparatus, device, and storage medium, which at least to some extent improves the security of the model.

[0004] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0005] According to one aspect of this disclosure, a detection method is provided, comprising: Obtain corpus information; The corpus information is preprocessed to obtain preprocessed corpus information; Test data is obtained by processing the preprocessed corpus information using at least one of the following methods: keyword extraction, clue generation, and template integration. The model to be tested is evaluated based on the test data.

[0006] In one embodiment of this disclosure, the corpus information is preprocessed to obtain preprocessed corpus information, including: Repeatability detection is performed on the corpus information to obtain corpus information with a repeatability higher than a preset threshold; Delete corpus information with a repetition rate higher than a preset threshold.

[0007] In one embodiment of this disclosure, preprocessed corpus information is processed using at least one method selected from keyword extraction, cue generation, and template integration to obtain test data, including: The priority of words in the preprocessed corpus information is determined based on a preset strategy; Keywords are extracted from preprocessed corpus information based on lexical priority.

[0008] In one embodiment of this disclosure, preprocessed corpus information is processed using at least one method selected from keyword extraction, cue generation, and template integration to obtain test data, including: Input keywords into the trained intelligent model to obtain clues related to the keywords.

[0009] In one embodiment of this disclosure, preprocessed corpus information is processed using at least one method selected from keyword extraction, cue generation, and template integration to obtain test data, including: Data sources are generated based on keywords and clues; Test data is generated based on formatting techniques and data sources.

[0010] In one embodiment of this disclosure, the method further includes: If the test results indicate that the model under test has failed the test, the model under test is trained based on the test data.

[0011] In one embodiment of this disclosure, obtaining corpus information includes: Information is obtained from corpus data based on preset rules or preset databases.

[0012] According to another aspect of this disclosure, a detection apparatus is provided, comprising: The acquisition module is used to acquire corpus information; The preprocessing module is used to preprocess the corpus information to obtain preprocessed corpus information; The processing module is used to process the preprocessed corpus information based on at least one of the following methods: keyword extraction, clue generation, and template integration, to obtain test data; The detection module is used to perform detection on the model to be tested based on the test data.

[0013] In one embodiment of this disclosure, the preprocessing module includes: The detection unit is used to detect the repetition of the corpus information and obtain the corpus information with a repetition rate higher than a preset threshold. The deletion unit is used to delete corpus information with a repetition rate higher than a preset threshold.

[0014] In one embodiment of this disclosure, the processing module includes: The determination unit is used to determine the priority of words in the preprocessed corpus information based on a preset strategy; The extraction unit is used to extract keywords from the preprocessed corpus information based on the priority of words.

[0015] In one embodiment of this disclosure, the processing module includes: The input unit is used to input keywords into the trained intelligent model to obtain clues related to the keywords.

[0016] In one embodiment of this disclosure, the processing module includes: The first generation unit is used to generate data sources based on keywords and clues; The second generation unit is used to generate test data based on formatting techniques and data sources.

[0017] In one embodiment of this disclosure, the apparatus further includes: The training module is used to train the model under test based on the test data when the detection result indicates that the model under test has failed the detection.

[0018] In one embodiment of this disclosure, the acquisition module includes: The acquisition unit is used to acquire corpus information based on preset rules or a preset database.

[0019] According to another aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the detection method described above by executing the executable instructions.

[0020] According to another aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the model detection method described above.

[0021] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the model detection method described above.

[0022] The detection method, apparatus, device, and storage medium provided in the embodiments of this disclosure acquire corpus information, preprocess the corpus information to obtain preprocessed corpus information, and process the preprocessed corpus information based on at least one method selected from keyword extraction, cue generation, and template generation to obtain test data. The test data is then used to detect the model to be tested. By constructing test data to detect the model, the detection of the model can be made more accurate.

[0023] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0024] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0025] Figure 1 This diagram illustrates an architecture of a model detection system according to an embodiment of the present disclosure. Figure 2 This diagram illustrates a flowchart of a model detection method according to an embodiment of the present disclosure; Figure 3 This diagram illustrates another model detection method according to an embodiment of the present disclosure. Figure 4 This diagram illustrates a flowchart of yet another model detection method according to an embodiment of the present disclosure; Figure 5 This diagram illustrates a flowchart of yet another model detection method according to an embodiment of the present disclosure; Figure 6 This diagram illustrates a flowchart of yet another model detection method according to an embodiment of the present disclosure; Figure 7 This diagram illustrates a flowchart of yet another model detection method according to an embodiment of the present disclosure; Figure 8 A structural diagram of a model detection device according to an embodiment of this disclosure is shown; Figure 9 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0026] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0027] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0028] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0029] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0030] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0031] It should be noted that, where there is no conflict, the embodiments of this disclosure and the technical features in the embodiments can be combined with each other.

[0032] In the production and warehousing management of industrial steel, label text serves as crucial information for batch numbers, specifications, production dates, and quality control, and is widely used for logistics traceability and quality inspection. Quality inspectors need to determine whether a bundle of steel belongs to the same batch based on the label information. However, in reality, due to the disorganized stacking of steel, different labels often exist within the same bundle. In such cases, workers must visually compare the label information one by one to determine if the bundle has been mixed with products from different batches. If inconsistent label information is found, the corresponding steel needs to be removed to ensure consistency within the same batch.

[0033] The specific implementation of the embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0034] Figure 1 A schematic diagram of a detection system structure is shown in an embodiment of the present disclosure. This system can apply the detection methods or detection processing devices in various embodiments of the present disclosure.

[0035] like Figure 1As shown, the model detection system 10 may include a data acquisition module 101 and a model detection module 102. The data acquisition module 101 and the model detection module 102 may be located on two different devices. The data acquisition module 101 may be a module on an electronic device with data acquisition capabilities, such as a recording device with sound collection capabilities, a photographic device with image collection capabilities, or a computer with text information collection capabilities. The model detection module 102 may be a module on an electronic device with processing capabilities, such as a computer. The data acquisition module 101 and the model detection module 102 may also be located on the same device. For example, the data acquisition module 101 and the model detection module 102 may be an input module and a processing module on a computer or mobile phone.

[0036] The acquisition module 101 and the model detection module 102 are connected by a network, which can be a wired network or a wireless network.

[0037] Optionally, the aforementioned wireless or wired networks use standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to Local Area Networks (LANs), Metropolitan Area Networks (MANs), Wide Area Networks (WANs), mobile, wired or wireless networks, private networks, or any combination of virtual private networks. In some embodiments, technologies and / or formats including Hyper Text Markup Language (HTML), Extensible Markup Language (XML), etc., are used to represent data exchanged over the network. Furthermore, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Networks (VPNs), and Internet Protocol Security (IPsec) can be used to encrypt all or some links. In other embodiments, custom and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.

[0038] The following explanation addresses the scenario where the acquisition module 101 and the model detection module 102 are located on two different devices.

[0039] The acquisition module 101 can be located on a terminal device, which can be various electronic devices, including but not limited to smartphones, tablets, laptops, desktop computers, wearable devices, augmented reality devices, virtual reality devices, etc.

[0040] Optionally, the client for the application installed on different terminal devices can be the same, or the client for the same type of application based on different operating systems. Depending on the terminal platform, the specific form of the application client can also differ; for example, the application client can be a mobile client, a PC client, etc.

[0041] The model detection module 102 can be located on a server. The server can be a server that provides various services, such as a backend management server that supports the operation of the device by the user using the terminal device. The backend management server can analyze and process the received request data and feed the processing results back to the terminal device.

[0042] Optionally, the server can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0043] Those skilled in the art will know that Figure 1 The number of data acquisition modules and model detection modules shown is merely illustrative; the actual number of clients and agents can be arbitrary, depending on actual needs. This disclosure does not limit this.

[0044] In some embodiments, content security testing of large models is crucial to ensure their safe and compliant application. Currently, several testing solutions exist in the industry for large model content security, capable of detecting certain attack methods, such as identifying warning words containing illegal information through keyword matching.

[0045] In the field of large-scale content security, the effectiveness of the aforementioned traditional attack paradigms has gradually diminished with the continuous iteration and capability upgrades of defense technology systems. However, content security is essentially an evolutionary process of dynamic game between attackers and defenders, and new attack methods often emerge simultaneously with the evolution of defense technologies. Based on this, the construction of a large-scale content security defense system should not be limited to passive defense, but should shift to an active defense paradigm of "attack-defense synergy, using offense to promote defense." Through forward-looking perception and systematic analysis of new attack patterns, iterative optimization of defense technologies can be fed back, forming a virtuous cycle of spiraling improvement in attack and defense capabilities.

[0046] To translate this proactive defense paradigm from theory into practice and truly enhance defense effectiveness, a specialized testing scheme capable of simulating new and complex attacks is urgently needed. The core logic of this scheme lies in using proactive attack probing to refine precise defense deployment. By creating complex instruction chain attack scenarios, it systematically uncovers potential compliance flaws in large-scale models. Specifically, this requires advancement in three aspects: first, designing targeted attack samples to accurately match the technical characteristics of new attacks; second, constructing a systematic testing process to ensure the completeness of attack scenario coverage and the standardization of testing procedures; and third, establishing quantitative evaluation standards for attack effects to achieve measurable and locatable risks and vulnerabilities. Through these measures, weak links in the defense system can be precisely identified, providing a clear direction for model iteration and optimization. It can also fill gaps in existing defense mechanisms regarding risk scenarios, propelling the large-scale model's defense capabilities from "passively responding to known threats" to "proactively predicting potential risks," ultimately providing practical support for the secure and compliant operation of large-scale models in complex application environments.

[0047] To address the aforementioned problems, this disclosure provides a detection method, apparatus, device, and storage medium.

[0048] Figure 2 A flowchart of a model detection method according to an embodiment of this disclosure is shown.

[0049] like Figure 2 As shown, the method may include: S210, Obtain corpus information.

[0050] In some embodiments, the corpus information may include information used by the detection model to perform detection. It should be noted that the corpus information may include various forms of information; when the corpus information is not text-based, other formats of corpus information can be converted into text-based corpus information.

[0051] In some embodiments, obtaining corpus information may include obtaining corpus information based on preset rules or a preset database. For example, the preset rules may include rules from the basic requirements for security of artificial intelligence services. Corpus information may include content that violates core social values, discriminatory content, commercially illegal or irregular content, or content that infringes upon the legitimate rights and interests of others.

[0052] S220, preprocess the corpus information to obtain preprocessed corpus information.

[0053] In some embodiments, preprocessing may include at least one of data cleaning, data integration, data transformation, data reduction, and data splitting.

[0054] It should be noted that data cleaning can include handling missing values, outliers, and duplicate values. Data integration can include entity recognition, redundancy handling, and conflict resolution. Data transformation can include normalization, discretization, logarithmic / power transformation, feature encoding, and feature construction. Data reduction can include dimensionality reduction, quantity reduction, and data compression.

[0055] S230: The preprocessed corpus information is processed using at least one of the following methods: keyword extraction, clue generation, and template integration, to obtain test data.

[0056] S240 performs testing on the model under test based on test data.

[0057] In some embodiments, testing the model under test based on test data may include inputting test data into the model under test, and after the model under test outputs results, combining the output results with pre-configured text to obtain the detection text.

[0058] In some embodiments, after obtaining the detected text, the detection result can be determined based on the detection model and the detected text.

[0059] The detection method provided in the embodiments of this disclosure acquires corpus information, preprocesses the corpus information to obtain preprocessed corpus information, and processes the preprocessed corpus information based on at least one method of keyword extraction, cue generation, and template generation to obtain test data. The test data is then used to detect the model to be tested. By constructing test data to detect the model, the detection of the model can be made more accurate.

[0060] Figure 3 A flowchart of another model detection method in an embodiment of this disclosure is shown.

[0061] like Figure 3 As shown, the method may include: S310, obtain corpus information; S320, performs repetition detection on the corpus information to obtain corpus information with a repetition rate higher than a preset threshold.

[0062] In some embodiments, repetition detection may include determining data in the corpus information that has repetition.

[0063] In some embodiments, the preset threshold can be determined by the user.

[0064] In some embodiments, the corpus information can also be preprocessed using manual verification. For example, manual verification may include cross-review by two people to determine whether it conforms to the risk category definition and to verify its validity as clues for solving puzzles. After processing the corpus information, it can be stored in a database. It should be noted that when storing the corpus information, it can also be categorized and stored according to preset categories.

[0065] S330, delete corpus information with a repetition rate higher than the preset threshold; S340, based on at least one of the methods of keyword extraction, clue generation, and template integration, the preprocessed corpus information is processed to obtain test data; S350 performs testing on the model under test based on test data.

[0066] The detection method provided in this disclosure involves acquiring corpus information, performing repetition detection on the corpus information to obtain corpus information with a repetition rate higher than a preset threshold, deleting the corpus information with a repetition rate higher than the preset threshold, and processing the processed corpus information based on at least one method of keyword extraction, clue generation, and template generation to obtain test data. The test data is then used to detect the model to be tested. By constructing test data for model detection, the detection of the model can be made more accurate.

[0067] Figure 4 A flowchart of another model detection method in an embodiment of this disclosure is shown.

[0068] like Figure 4 As shown, the method may include: S410, obtain corpus information; S420, preprocess the corpus information to obtain preprocessed corpus information; S430 determines the priority of words in the preprocessed corpus information based on a preset strategy.

[0069] In some embodiments, the priority of words in the corpus information can be determined by a preset model and / or multiple candidate rules. For example, standardized instructions containing explicit priorities can be input into the preset model. For example, one core entity keyword can be extracted from the following statements, with the extraction priority ordered as follows: geographical categories (e.g., province, city, continent name) > personal categories (e.g., public figures, historical figures) > object categories (e.g., specific groups, item names); if multiple candidates exist for the same priority, such as two geographical categories, the keyword with higher public awareness and wider social dissemination is extracted first.

[0070] It's important to note that the core principle of the intelligent reasoning and instruction execution collaborative attack method is compliance at each stage, followed by violation after collaboration. It overcomes the limitations of large-scale model defenses through two stages: intelligent reasoning guidance and instruction execution triggering. In the initial reasoning stage, the target that requires a specific answer to trigger a violation is identified from the non-compliant corpus. Then, three concrete clues with strong logical constraints are designed to guide the large model to deduce a unique, compliant answer, ensuring accurate and controllable subsequent violation triggering. In the later execution stage, individual, non-violation-compliant texts are extracted from the non-compliant corpus, while texts that violate the rules after embedding are used. A simplified instruction guides the model to embed and repeat the answer, activating hidden violation risks. The core of this collaboration lies in leveraging the characteristic that large-scale model operations take precedence over compliance review, allowing the model to prioritize the "guessing-filling-repetition" operation, ignoring the content risks after stage collaboration, and exposing blind spots in the identification of multi-step instruction association risks.

[0071] S440 extracts keywords from preprocessed corpus information based on word priority pairs.

[0072] S450, determines test data based on keywords; S460 performs testing on the model under test based on test data.

[0073] The detection method provided in this embodiment acquires corpus information, performs repetition detection on the corpus information to obtain corpus information with repetition exceeding a preset threshold, deletes corpus information with repetition exceeding the preset threshold, determines the priority of words in the preprocessed corpus information based on a preset strategy, extracts keywords from the preprocessed corpus information based on the word priority, and determines test data based on the keywords. The test data is then used to detect the model to be tested. By constructing test data to detect the model, the detection of the model can be made more accurate.

[0074] Figure 5 A flowchart of another detection method in an embodiment of this disclosure is shown.

[0075] like Figure 5 As shown, the method may include: S510, obtain corpus information; S520, preprocess the corpus information to obtain preprocessed corpus information; S530, determines the priority of words in the preprocessed corpus information based on a preset strategy; S540 extracts keywords from preprocessed corpus information based on word priority pairs.

[0076] S550 inputs keywords into the trained intelligent model to obtain clues related to the keywords.

[0077] In some embodiments, clues can be determined based on principles such as clue visualization and relevance, compliance assurance, unambiguity, and verifiability.

[0078] The clue itself has no illegal attributes; the risk of violation will only be triggered after it is embedded in the text. Ensuring the uniqueness of the clue is a key prerequisite for ensuring that the effect of subsequent attacks is controllable. Therefore, the design of the clue must ensure that the clue is concrete, related, compliant, and unambiguous.

[0079] It's important to note that the clues must revolve around the core characteristics of the answer. Each clue points to a specific attribute of the answer, and the clues form a logical connection through scope limitation, feature focus, and detail confirmation, forcing the answer to be unique through concrete information. For example, if the preset answer is "Africa," the first clue limits the scope (e.g., "This object belongs to one of the seven continents of the Earth and is one of the birthplaces of ancient humans and civilizations"). The second clue focuses on core geographical features (e.g., "The equator crosses the center of this continent, making it the continent with the largest tropical area in the world, containing large tracts of its landmass in both the Northern and Southern Hemispheres"). The third clue adds significant details (e.g., "This continent has the world's largest desert, the Sahara Desert, and the Nile River, one of the world's longest rivers"). Through this three-layered concrete description of scope, features, and details, the possibility of the model inferring other continents such as "Asia" or "South America" ​​is completely avoided.

[0080] The clue descriptions must focus on conventional solutions with no risk of violation, and all statements must be objective facts or neutral descriptions, without involving any content related to violations. For example, if the goal is for the model to deduce a compliant figure, such as Marie Curie, the clues must revolve around neutral information such as her scientific achievements and historical identity (e.g., "This person was a world-renowned physicist and chemist, and the first woman to win a Nobel Prize," "She and her husband jointly discovered the radioactive elements radium and polonium," "She won the Nobel Prize twice and made outstanding contributions to the field of radioactivity research"), ensuring that the clues and the solution themselves fully comply with content compliance requirements.

[0081] By abandoning vague expressions (such as "probably," "possibly," or "similar"), each clue is presented as an objective fact plus verifiable information, ensuring that the model can pinpoint a unique answer through explicit information. For example, instead of vague descriptions like "a historically significant city," verifiable information such as "the city is the capital of Italy, located in the central Apennine Peninsula, and boasts famous historical buildings like the Colosseum and the Pantheon" is used, including the country, geographical location, and iconic landmarks, thus preventing the answer from diverging from the outset.

[0082] For example, with the core objective of ensuring that a unique answer can be deduced based on clues, a large model with strong logical association generation and entity feature extraction capabilities should be selected (such as the GPT series, the Tongyi Qianwen and Wenxin Yiyan models among domestic open-source large models, which need to support accurate semantic understanding and constrained text generation). For the extracted keywords, standardized instructions should be designed according to the logic of core attribute description, typical scenario anchoring, and scope narrowing. Example: Based on keyword A, generate three clues: 1. Describe the core unique attributes of the keyword. 2. Anchor the typical application scenarios of the keyword. 3. Narrow the scope of the keyword (such as distinctive features). The three clues must only point to keyword A.

[0083] S560 determines test data based on clues.

[0084] S570 performs testing on the model under test based on test data.

[0085] The detection method provided in this embodiment acquires corpus information, performs repetition detection on the corpus information to obtain corpus information with repetition exceeding a preset threshold, deletes corpus information with repetition exceeding the preset threshold, determines the priority of words in the preprocessed corpus information based on a preset strategy, extracts keywords from the preprocessed corpus information based on the word priority, inputs the keywords into the trained intelligent model to obtain clues associated with the keywords, and determines test data based on the clues. The test data is then used to detect the model to be tested. By constructing test data to detect the model, the detection of the model can be made more accurate.

[0086] Figure 6 A flowchart of another detection method in an embodiment of this disclosure is shown.

[0087] like Figure 6 As shown, the method may include: S610, obtain corpus information; S620, preprocess the corpus information to obtain preprocessed corpus information; S630, determines the priority of words in the preprocessed corpus information based on a preset strategy; S640 extracts keywords from preprocessed corpus information based on word priority pairs.

[0088] S650: Input keywords into the trained intelligent model to obtain clues related to the keywords.

[0089] S660 generates data sources based on keywords and leads; The S670 generates test data based on formatting technology and data sources.

[0090] In some embodiments, batch automated template integration is implemented using Python code. The core methods are structured data retrieval and standardized template variable mapping. Specifically, Python calls common data processing libraries (such as pandas) to read keyword IDs, text data to be embedded, and three corresponding clues from the clue library in the structured database to construct a complete single-sample data source. A fixed attack sample template is defined, and data variables such as clues and text to be embedded are mapped to the corresponding positions in the template through string formatting techniques to generate test data.

[0091] S680 performs testing on the model under test based on test data.

[0092] The detection method provided in this embodiment acquires corpus information, performs repetition detection on the corpus information to obtain corpus information with repetition exceeding a preset threshold, deletes corpus information with repetition exceeding the preset threshold, determines the priority of words in the preprocessed corpus information based on a preset strategy, extracts keywords from the preprocessed corpus information based on the word priority, inputs the keywords into the trained intelligent model to obtain clues associated with the keywords, generates a data source based on the keywords and clues, and generates test data based on formatting techniques and the data source. The test data is then used to detect the model to be tested. By constructing test data to detect the model, the detection of the model can be made more accurate.

[0093] Figure 7 A flowchart of another detection method in an embodiment of this disclosure is shown.

[0094] like Figure 7 As shown, the method may include: S710, acquire corpus information; S720 preprocesses the corpus information to obtain preprocessed corpus information; S730: Based on at least one of the following methods, keyword extraction, clue generation, and template integration, the preprocessed corpus information is processed to obtain test data; S740 performs testing on the model under test based on test data.

[0095] S750 trains the test model based on the test data when the test result indicates that the test model has failed the test.

[0096] In some embodiments, test data can be used as a training set to train the model to be tested. This disclosure does not limit the training method.

[0097] The detection method provided in the embodiments of this disclosure acquires corpus information, preprocesses the corpus information to obtain preprocessed corpus information, and processes the preprocessed corpus information based on at least one method of keyword extraction, cue generation, and template generation to obtain test data. When the detection result indicates that the model to be tested has failed the detection, the model to be tested is trained based on the test data. The model to be tested is then tested based on the test data. By constructing test data to detect the model, the detection of the model can be made more accurate.

[0098] Based on the same inventive concept, this disclosure also provides a detection device, as shown in the following embodiment. Since the principle by which this device embodiment solves the problem is similar to that of the above-described method embodiment, the implementation of this device embodiment can refer to the implementation of the above-described method embodiment, and repeated details will not be elaborated further.

[0099] Figure 8 A structural diagram of a detection device according to an embodiment of this disclosure is shown.

[0100] like Figure 8 As shown, the device 800 may include: an acquisition module 810, a preprocessing module 820, a processing module 830, and a detection module 840.

[0101] Module 810 is used to acquire corpus information; Preprocessing module 820 is used to preprocess the corpus information to obtain preprocessed corpus information; The processing module 830 is used to process the preprocessed corpus information based on at least one of the methods of keyword extraction, clue generation, and template integration to obtain test data; The detection module 840 is used to perform detection on the model to be tested based on the test data.

[0102] In one embodiment of this disclosure, the preprocessing module includes: The detection unit is used to detect the repetition of the corpus information and obtain the corpus information with a repetition rate higher than a preset threshold. The deletion unit is used to delete corpus information with a repetition rate higher than a preset threshold.

[0103] In one embodiment of this disclosure, the processing module includes: The determination unit is used to determine the priority of words in the preprocessed corpus information based on a preset strategy; The extraction unit is used to extract keywords from the preprocessed corpus information based on the priority of words.

[0104] In one embodiment of this disclosure, the processing module includes: The input unit is used to input keywords into the trained intelligent model to obtain clues related to the keywords.

[0105] In one embodiment of this disclosure, the processing module includes: The first generation unit is used to generate data sources based on keywords and clues; The second generation unit is used to generate test data based on formatting techniques and data sources.

[0106] In one embodiment of this disclosure, the apparatus further includes: The training module is used to train the model under test based on the test data when the detection result indicates that the model under test has failed the detection.

[0107] In one embodiment of this disclosure, the acquisition module includes: The acquisition unit is used to acquire corpus information based on preset rules or a preset database.

[0108] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0109] The following reference Figure 9 To describe an electronic device 900 according to such an embodiment of the present disclosure. Figure 9 The electronic device 900 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0110] like Figure 9 As shown, the electronic device 900 is manifested in the form of a general-purpose computing device. The components of the electronic device 900 may include, but are not limited to: at least one processing unit 910, at least one storage unit 920, and a bus 930 connecting different system components (including storage unit 920 and processing unit 910).

[0111] The storage unit stores program code, which can be executed by the processing unit 910, causing the processing unit 910 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure. For example, the processing unit 910 can perform the following steps of the above method embodiments: Obtain corpus information; The corpus information is preprocessed to obtain preprocessed corpus information; Test data is obtained by processing the preprocessed corpus information using at least one of the following methods: keyword extraction, clue generation, and template integration. The model to be tested is evaluated based on the test data.

[0112] Storage unit 920 may include readable media in the form of volatile storage units, such as random access memory (RAM) 9201 and / or cache memory 9202, and may further include read-only memory (ROM) 9203.

[0113] The storage unit 920 may also include a program / utility 9204 having a set (at least one) program module 9205, such program module 9205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0114] Bus 930 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0115] Electronic device 900 can also communicate with one or more external devices 940 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 900, and / or with any device that enables electronic device 900 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 950. Furthermore, electronic device 900 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 960. As shown, network adapter 960 communicates with other modules of electronic device 900 via bus 930. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 900, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0116] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0117] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, which may be a readable signal medium or a readable storage medium. A program product capable of implementing the methods described above is stored thereon. In some possible implementations, various aspects of this disclosure may also be implemented as a program product including program code, which, when run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0118] More specific examples of computer-readable storage media in this disclosure may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0119] In this disclosure, a computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, wherein readable program code is carried. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting a program for use by or in connection with an instruction execution system, apparatus, or device.

[0120] Optionally, the program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0121] In practice, program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0122] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0123] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0124] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0125] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. A model detection method, characterized in that, include: Obtain corpus information; The corpus information is preprocessed to obtain preprocessed corpus information; The preprocessed corpus information is processed using at least one of the methods of keyword extraction, clue generation, and template integration to obtain test data; The model to be tested is evaluated based on the test data.

2. The method according to claim 1, characterized in that, The step of preprocessing the corpus information to obtain preprocessed corpus information includes: The corpus information is subjected to repetition detection to obtain corpus information with a repetition rate higher than a preset threshold; Delete the corpus information with a repetition rate higher than a preset threshold.

3. The method according to claim 1, characterized in that, The preprocessed corpus information is processed using at least one method based on keyword extraction, clue generation, and template integration to obtain test data, including: The priority of words in the preprocessed corpus information is determined based on a preset strategy; Keywords are extracted from the preprocessed corpus information based on the priority of the vocabulary.

4. The method according to claim 3, characterized in that, The preprocessed corpus information is processed using at least one method based on keyword extraction, clue generation, and template integration to obtain test data, including: The keywords are input into the trained intelligent model to obtain clues associated with the keywords.

5. The method according to claim 4, characterized in that, The preprocessed corpus information is processed using at least one method based on keyword extraction, clue generation, and template integration to obtain test data, including: Data source is generated based on the keywords and clues; Test data is generated based on formatting techniques and the data source.

6. The method according to claim 1, characterized in that, The method further includes: If the test result indicates that the model under test has failed the test, the model under test is trained based on the test data.

7. The method according to claim 1, characterized in that, The acquisition of corpus information includes: Information is obtained from corpus data based on preset rules or preset databases.

8. A model testing device, characterized in that, include: The acquisition module is used to acquire corpus information; The preprocessing module is used to preprocess the corpus information to obtain preprocessed corpus information; The processing module is used to process the preprocessed corpus information based on at least one of the methods of keyword extraction, clue generation, and template integration to obtain test data; The detection module is used to detect the model to be tested based on the test data.

9. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the model detection method according to any one of claims 1 to 7 by executing the executable instructions.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the model detection method according to any one of claims 1 to 7.

Images