Test methods, apparatuses, devices, storage media, and products

Abstract

The application discloses a test method, device, equipment, storage medium and product, and belongs to the technical field of artificial intelligence. The method comprises the following steps: obtaining a test sample; performing target identification processing on the test sample to obtain identification information corresponding to the original sample and the adversarial sample respectively; obtaining sample information corresponding to the original sample and the adversarial sample respectively; determining at least one test performance index corresponding to the target identification processing based on the sample information and the identification information; and determining a test result corresponding to the target identification processing based on the at least one test performance index. The embodiments of the application can be applied to various scenes such as cloud technology, artificial intelligence, intelligent transportation, auxiliary driving and vehicle-mounted application. The sample information and the identification information corresponding to the original sample and the adversarial sample are combined to determine the at least one test performance index to comprehensively measure the robustness of the target identification processing, so that a comprehensive and accurate test result is obtained, and the accuracy of the test performance index, the accuracy and the scientificity of the test result are improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a testing method, apparatus, device, storage medium, and product. Background Technology

[0002] With the continuous advancement of artificial intelligence technology, information recognition and processing technologies based on deep learning have developed rapidly, especially speech recognition technology, which is widely used in real-time communication, security protection, and autonomous driving. Currently, an increasing number of applications offer intelligent voice interaction services to users.

[0003] However, in practical applications, malicious attacks targeting information recognition and processing technologies have emerged. One type of malicious attack is the adversarial example attack, where attackers add imperceptible interference information to the original information input, leading to incorrect recognition results. Therefore, it is necessary to test the robustness of the aforementioned information recognition and processing technologies.

[0004] In related technologies, the aforementioned information identification and processing techniques are used to identify and process generated adversarial examples, thereby determining the success rate of attacks launched by these adversarial examples, and using this success rate to measure the robustness of the information identification and processing techniques. However, in these related technologies, the test indicators for measuring the robustness of information identification and processing techniques are singular, and the accuracy of the test results is low. Summary of the Invention

[0005] This application provides a testing method, apparatus, device, storage medium, and product that can determine at least one test performance index, and these are test performance indicators in different dimensions, thereby solving the problem of low accuracy of test results due to a single test index and improving the accuracy and scientific nature of the test results.

[0006] According to one aspect of the embodiments of this application, a testing method is provided, the method comprising:

[0007] Obtain test samples, which include original samples and adversarial samples corresponding to the original samples;

[0008] The test sample is subjected to target recognition processing to obtain the recognition information corresponding to the original sample and the adversarial sample respectively;

[0009] Obtain the sample information corresponding to the original sample and the adversarial sample respectively;

[0010] Based on the sample information and the recognition information, at least one test performance index is determined for the target recognition process. The at least one test performance index characterizes the recognition accuracy and recognition stability of the target recognition process from at least one dimension.

[0011] determine a test result corresponding to the target recognition processing based on the at least one test performance indicator.

[0012] According to an aspect of an embodiment of the present application, a test device is provided, and the device comprises:

[0013] a sample acquisition module configured to acquire a test sample, the test sample comprising an original sample and an adversarial sample corresponding to the original sample;

[0014] a sample identification module configured to perform target recognition processing on the test sample to obtain identification information corresponding to the original sample and the adversarial sample, respectively;

[0015] an information acquisition module configured to acquire sample information corresponding to the original sample and the adversarial sample, respectively;

[0016] a test indicator determination module configured to determine at least one test performance indicator corresponding to the target recognition processing based on the sample information and the identification information, the at least one test performance indicator representing the identification accuracy and the identification stability of the target recognition processing from at least one dimension;

[0017] a test result determination module configured to determine a test result corresponding to the target recognition processing based on the at least one test performance indicator.

[0018] According to an aspect of an embodiment of the present application, a computer device is provided, and the computer device comprises a processor and a memory, the memory storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by the processor to implement the test method described above.

[0019] According to an aspect of an embodiment of the present application, a computer readable storage medium is provided, and the storage medium stores at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by a processor to implement the test method described above.

[0020] According to an aspect of an embodiment of the present application, a computer program product is provided, and the computer program product comprises computer instructions stored in a computer readable storage medium, a processor of a computer device reading the computer instructions from the computer readable storage medium, the processor executing the computer instructions to cause the computer device to perform to implement the test method described above.

[0021] The technical solutions provided by the embodiments of the present application can bring the following beneficial effects:

[0022] By performing target identification processing on the original sample and the adversarial sample in the test sample, the respective identification information of the original sample and the adversarial sample can be obtained. In order to make the test result of the above target identification processing more accurate and avoid the influence of the sample difference between the two samples on the test result, the two kinds of identification information obtained above are combined with the original sample information of the original sample and the adversarial sample respectively, to determine at least one test performance indicator that can represent the identification accuracy and identification stability of the above target identification processing, and is a test performance indicator in different dimensions, which effectively improves the accuracy of the test performance indicator. Finally, at least one test performance indicator in at least one dimension is used to comprehensively measure the robustness of the target identification processing, so that a more accurate, reliable and comprehensive test result can be obtained, solving the problem of low accuracy of test results caused by single test indicator, and improving the accuracy and scientificity of the test result. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0024] Figure 1 is a schematic diagram of an application running environment provided by an embodiment of the present application;

[0025] Figure 2 is a flow of a test method provided by an embodiment of the present application Figure One ;

[0026] Figure 3 is a flow of a test method provided by an embodiment of the present application Figure Two ;

[0027] Figure 4 is a flow of a test method provided by an embodiment of the present application Figure Three ;

[0028] Figure 5 An exemplary technical architecture diagram of a speech recognition model robustness evaluation platform is shown.

[0029] Figure 6 is a block diagram of a test device provided by an embodiment of the present application;

[0030] Figure 7 is a structural block diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0031] The test method provided by the embodiments of the present application relates to artificial intelligence technology, which is briefly described below to facilitate understanding by those skilled in the art.

[0032] Artificial intelligence (AI) is the use of digital computers or digital computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making. For example, using artificial intelligence technology to recognize and process multimedia information, and performing and completing related identification tasks, thereby providing information recognition services for users.

[0033] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software technologies. Artificial intelligence basic technologies generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, autonomous driving, intelligent transportation, etc.

[0034] The key technologies of speech technology include automatic speech recognition (ASR) technology, speech synthesis technology and voiceprint recognition technology. Letting computers listen, see, speak and feel is the development direction of future human-computer interaction, and voice has become one of the most promising human-computer interaction methods in the future. Among them, automatic speech recognition technology is a technology that converts speech into text, and is a multidisciplinary field closely related to acoustics, phonetics, linguistics, digital signal processing theory, information theory, computer science and many other disciplines. Using the above speech technology, the collected or acquired speech information can be recognized, and related operations can be performed according to the recognition result. For example, collecting the speech information generated by the user and performing speech recognition processing, in the case of including a control instruction for a target device in the speech recognition result, the target device can be executed according to the recognized control instruction,

[0035] Natural Language Processing (NLP) is an important direction in the field of computer science and artificial intelligence. It studies various theories and methods that can realize effective communication between people and computers using natural language. Natural Language Processing is a science that integrates linguistics, computer science, and mathematics. Therefore, the research in this field will involve natural language, i.e. the language used in daily life, so it is closely related to the study of linguistics. Natural Language Processing technology usually includes text processing, semantic understanding, machine translation, robot question and answer, knowledge graph and other technologies. By using the above natural language processing technology, the text information in the above recognition result can be analyzed and processed to determine the user's intention, and then perform related operations or reply related information.

[0036] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, and other disciplines. It is a specialized study of how computers simulate or implement human learning behavior to acquire new knowledge or skills, reorganize existing knowledge structure to continuously improve their performance. Machine Learning is the core of artificial intelligence and the fundamental approach to making computers intelligent. Its applications are widespread in various fields of artificial intelligence. Machine Learning and Deep Learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and rule-based learning.

[0037] With the research and progress of artificial intelligence technology, artificial intelligence technology has been researched and applied in many fields, such as common smart home, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned vehicles, autonomous vehicles, drones, robots, smart medical care, smart customer service, Internet of Vehicles, autonomous driving, intelligent transportation, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0038] The scheme provided by the embodiments of the present application relates to voice technology, natural language processing, machine learning and other technologies of artificial intelligence. At the same time, the embodiments of the present application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, intelligent transportation, assisted driving, vehicle-mounted application, etc. In order to make the purpose, technical scheme and advantages of the present application clearer, the embodiments of the present application will be described in further detail below with reference to the drawings.

[0039] Please refer to Figure 1 which shows a schematic diagram of an application running environment provided by an embodiment of the present application. The application running environment can include a terminal 10 and a server 20.

[0040] The terminal 10 includes, but is not limited to, a mobile phone, a computer, a smart voice interaction device, a smart home appliance, a vehicle-mounted terminal, a game console, an e-book reader, a multimedia playing device, a wearable device, and the like. The terminal 10 can install a client of an application program.

[0041] In the embodiments of the present application, the application program can be any application program capable of providing an information recognition service. Typically, the information recognition service is a speech recognition service, an image recognition service, and a text recognition service. Of course, in addition to the speech recognition service, the image recognition service, and the text recognition service, the information recognition service also includes a service of recognizing other types of media information, which is not limited in the embodiments of the present application. Correspondingly, the application program including the various information recognition services includes, but is not limited to, a smart voice interaction application program, a virtual assistant application program, a device control application program, a system application program, an auxiliary driving application program, a map navigation application program, a social application program, an interactive entertainment application program, a news application program, a browser application program, a shopping application program, a content sharing application program, a virtual reality (VR) application program, an augmented reality (AR) application program, and the like, which is not limited in the embodiments of the present application. In addition, for different application programs, the information recognition services provided by the application programs are different, and the corresponding functions are also different, which can be pre-configured according to actual needs, which is not limited in the embodiments of the present application. Optionally, the terminal 10 runs the client of the application program.

[0042] The server 20 is configured to provide a background service for the client of the application program in the terminal 10. For example, the server 20 can be a background server of the application program. The server 20 can be a physical server, a server cluster composed of multiple physical servers, or a distributed system, and can also be a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms. Optionally, the server 20 simultaneously provides a background service for the application programs in multiple terminals 10.

[0043] Optionally, the terminal 10 and the server 20 can communicate with each other through the network 30. The terminal 10 and the server 20 can be directly or indirectly connected through wired or wireless communication, which is not limited in the present application.

[0044] Please refer to Figure 2 which shows a flow of a test method provided by an embodiment of the present applicationFigure One The method can be applied to a computer device, which refers to an electronic device with data calculation and processing capability. For example, the execution subject of each step can be a terminal 10 or a server 20 in the application program running environment as shown in the figure. The method can include the following steps (210-250). Figure 1

[0045] Step 210, obtaining a test sample.

[0046] The test sample includes an original sample and an adversarial sample corresponding to the original sample.

[0047] The original sample refers to original multimedia information in a data set. The data set includes but is not limited to an image data set, an audio data set, a video data set, and a text data set. Correspondingly, the media type of the original sample includes but is not limited to images, audio, video, and text, and the data format of the original sample can be any format, which is not limited in the embodiments of the present application.

[0048] An adversarial example refers to an input sample formed by deliberately adding subtle interference in a data set, which can cause the model to give an incorrect output with high confidence. The adversarial example makes subtle modifications to normal input information in a way that is imperceptible to humans, so that the artificial intelligence model makes an incorrect classification on the modified input. The adversarial example first appeared in the image field. The existence of the adversarial example indicates that the model tends to rely on unreliable features to maximize performance, and it is effective for deep learning systems related to text, images, and audio / video.

[0049] In an exemplary embodiment, the adversarial sample is generated based on the original sample and interference information, and is used to implement adversarial attacks on the target recognition processing to test the robustness of the target recognition processing. Therefore, to generate the adversarial sample in the test sample, as shown in the figure, the implementation process of step 210 includes the following steps (211-214), Figure 3 Figure 3 The figure shows the flow of the test method provided by an embodiment of the present application Figure Two .

[0050] Step 211, obtaining sample generation parameters.

[0051] ​​The sample generation parameters include, but are not limited to, sample dataset parameters, adversarial perturbation mode parameters, recognition and processing parameters, interference information generation parameters, and interference intensity parameters. The sample dataset parameters are used to identify and distinguish different sample datasets; the adversarial perturbation mode parameters are used to identify and distinguish different adversarial perturbation modes; the recognition and processing parameters are used to identify and distinguish different recognition and processing methods; the interference information generation parameters are used to identify and distinguish different interference information generation methods; and the interference intensity parameters are used to identify and distinguish different interference intensities.

[0052] Step 212: Obtain the original sample.

[0053] In one possible implementation, original samples are read or downloaded from a database. The database includes at least one pre-defined dataset. Optionally, each dataset corresponds to different sample dataset parameters. A terminal or server can obtain the pre-defined sample dataset parameters, determine a target dataset corresponding to the pre-defined sample dataset parameters, and use the data samples in the target dataset as the original samples.

[0054] In practical applications, at least one sample dataset should be selected as the test dataset. Alternatively, multiple or all sample datasets can be selected depending on the testing requirements. In one example, the preset sample dataset parameters are... , … , , … These are the identifier parameters corresponding to each selected target dataset. Accordingly, the target datasets include... ,in, , Let be the number of sample datasets in the database. Correspondingly, the number of original samples included in the target dataset can be denoted as . In some application scenarios, a small number of test samples is meaningless for testing the robustness of the model; therefore, the above-mentioned number of samples can be set. The minimum value, for example It is 100, and the maximum value is the number of all data samples in the entire dataset.

[0055] In speech recognition scenarios, the aforementioned database can be a speech database. Speech databases contain open-source, multilingual speech samples with various characteristics, including but not limited to Mozilla Common Voice (a public speech dataset), Tatoeba (Tatoeba is the name of an online database), VOiCES Dataset (a speech dataset), VoxCeleb (a large human voice recognition dataset), and other datasets.

[0056] Step 213, generating the identification interference information under at least one interference scene according to the sample generation parameter.

[0057] By setting different sample generation parameters, the processing device can generate identification interference information under different interference scenes or different identification interference information under the same interference scene according to different sample generation parameters.

[0058] In an example embodiment, the sample generation parameter includes at least one of an adversarial perturbation mode parameter, an identification processing parameter, and an interference information generation parameter. Accordingly, as shown in Figure 4 The implementation process of step 213 includes the following steps (2131-2133), Figure 4 The flow of the test method provided by an embodiment of the application is shown Figure Three .

[0059] Step 2131, determining an adversarial perturbation mode based on the adversarial perturbation mode parameter.

[0060] In specific applications, different adversarial perturbation mode parameters can be set to represent different adversarial perturbation modes, and the setting rules of the adversarial perturbation mode parameter are not limited in the embodiments of the application. The above-mentioned adversarial perturbation mode can be modularly expanded according to specific application scenarios to enrich the test system.

[0061] In a possible implementation, the adversarial perturbation mode parameter includes a first adversarial perturbation mode parameter, a second adversarial perturbation mode parameter, a third adversarial perturbation mode parameter, and a fourth adversarial perturbation mode parameter, which correspond to a first adversarial perturbation mode, a second adversarial perturbation mode, a third adversarial perturbation mode, and a fourth adversarial perturbation mode, respectively. In an example, the adversarial perturbation mode parameter is {1, 2, 3, 4}, where 1, 2, 3, and 4 are the first adversarial perturbation mode parameter, the second adversarial perturbation mode parameter, the third adversarial perturbation mode parameter, and the fourth adversarial perturbation mode parameter, respectively. The explanations of the above-mentioned four adversarial perturbation modes can be referred to the specific content below.

[0062] If the obtained sample generation parameter includes at least one of the first adversarial perturbation mode parameter, the second adversarial perturbation mode parameter, the third adversarial perturbation mode parameter, and the fourth adversarial perturbation mode parameter, the adversarial perturbation mode includes the corresponding adversarial perturbation mode.

[0063] Step 2132, obtaining an identification result of the original sample after the identification processing corresponding to the identification processing parameter.

[0064] Optionally, the first adversarial perturbation mode refers to a test mode representing an adversarial attack scenario. The adversarial attack scenario refers to a scenario in which an adversarial sample generated by an adversarial attack algorithm attacks an information recognition service. In specific applications, the adversarial attack scenario is usually a malicious attack scenario.

[0065] The recognition processing parameter is used to identify and distinguish different recognition processing modes. By setting different recognition processing parameters, the recognition results of the original sample after different recognition processing can be obtained. The recognition results of the original sample after different recognition processing can be used to generate different recognition interference information. By using the recognition results of the original sample after other recognition processing, more abundant recognition interference information can be generated to test the target recognition processing, and the robustness of the target recognition processing can be evaluated from more angles.

[0066] In a possible implementation, the recognition processing is implemented by an intelligent recognition model. The intelligent recognition model is a machine learning model trained according to training samples and corresponding sample labels, and is used to implement a specific recognition task. Accordingly, the recognition processing parameter is used to determine the intelligent recognition model, and different recognition processing parameters correspond to different intelligent recognition models. The recognition result of the original sample after the recognition processing corresponding to the recognition processing parameter can be replaced by the recognition result of the original sample on the intelligent recognition model corresponding to the recognition processing parameter.

[0067] In an example embodiment, the recognition processing includes a target recognition processing, which refers to an information recognition processing to be tested. Accordingly, the intelligent recognition model implementing the target recognition processing is a target recognition model. The intelligent recognition model corresponding to the recognition processing parameter can include the target recognition model, or can include other intelligent recognition models in addition to the target recognition model. Optionally, according to the recognition processing parameter, the intelligent recognition model corresponding to the recognition processing parameter is determined from an intelligent recognition model library. The intelligent recognition model library includes at least one intelligent recognition model. In a speech recognition scenario, the intelligent recognition model library is an intelligent speech model library, and the intelligent speech model library includes but is not limited to common speech models, such as Google Voice, DeepSpeech, Kaldi ASR, and other automatic speech recognition models.

[0068] In a possible implementation, each intelligent recognition model in the intelligent recognition model library corresponds to a unique recognition processing parameter. By setting the recognition processing parameter, multiple or all intelligent recognition models in the intelligent recognition model library can be selected, and the recognition results of the original sample on the selected intelligent recognition models can be used to generate different recognition interference information.

[0069] In one example, the set of identification processing parameters are The selected intelligent identification model includes , wherein are different identification processing parameters corresponding to the selected intelligent identification model, and can also be denoted as the name identifiers of the respective intelligent identification models, and i is a model sequence identifier. The number of models in the above intelligent identification model library can be denoted as . Correspondingly, the above .

[0070] In some application scenarios, the above intelligent identification model library is a model library composed of other intelligent identification models in addition to the target identification model. When testing the robustness of the target identification processing, i.e., testing the robustness of the target identification model, other intelligent models in the intelligent identification model library can not be selected, and the identification results of the original sample on the other intelligent identification models are not used to generate identification interference information. Without selecting other intelligent models in the intelligent identification model library, i.e., the above , the identification results of the original sample on the target identification model to be tested can be used to generate corresponding identification interference information. Optionally, the identification interference information generated by the identification results of the original sample on the target identification model to be tested is identification interference information generated based on a black box attack algorithm.

[0071] Embodiments of the present application design a plurality of identification interference information in an adversarial attack scenario by using the identification results of the original sample after a plurality of identification processes, to implement adversarial attacks on the target identification processing to be tested, so that the test system is more complete, and is conducive to ensuring the accuracy and reliability of the test results.

[0072] Step 2133, performing interference information generation processing corresponding to the interference information generation parameters on the identification results and the original sample, to obtain identification interference information in an adversarial attack scenario.

[0073] The above identification interference information is disturbance information used to interfere with the target identification processing.

[0074] The above interference information generation parameters are used to identify and distinguish different interference information generation methods. The processing method of the above interference information generation processing is determined according to the adversarial sample generation algorithm adopted by the system, so that the adversarial sample generation algorithm adopted by the system can be determined according to the above interference information generation parameters, and the above interference information generation method can be determined.

[0075] In a possible implementation, to improve the robustness of the test target recognition processing, the interference information generation processing described above needs to be performed by selecting at least one of the adversarial sample generation algorithms from the adversarial sample generation algorithm library. Therefore, the interference information generation parameter can be obtained, at least one of the adversarial sample generation algorithms corresponding to the obtained interference information generation parameter can be determined, and the recognition result can be analyzed and processed according to the adversarial sample generation algorithm to generate the recognition interference information corresponding to the original sample, that is, the recognition interference information in the adversarial attack scenario described above. Different interference information generation parameters correspond to different adversarial sample generation algorithms, and each adversarial sample generation algorithm corresponds to a unique interference information generation parameter.

[0076] In an example, the pre-set interference information generation parameter obtained is , which respectively represents the algorithm identifiers of different adversarial sample generation algorithms, where is an algorithm serial number identifier. Correspondingly, according to the interference information generation parameter described above, the selected set of adversarial sample generation algorithms is determined as . Optionally, the determined adversarial sample generation algorithm is an algorithm in the adversarial sample generation algorithm library. The number of algorithms in the adversarial sample generation algorithm library can be denoted as , and therefore satisfies When , at least a black-box adversarial sample generation method is selected, and the recognition result of the original sample on the target recognition model to be tested is analyzed using the black-box adversarial sample generation method to determine the recognition interference information corresponding to the original sample.

[0077] Optionally, the adversarial sample generation algorithm includes a black-box test-based adversarial sample generation algorithm. The black-box test refers to a test in which an attacker has no knowledge of the internal structure, training parameters, defense methods, and the like of the model to be attacked, and can only interact with the model through the output to achieve the adversarial attack. Since the adversarial attack on the target recognition processing in a real scenario is often a black-box condition, the adversarial sample generation algorithm based on the black-box test is used to generate the recognition interference information, which can effectively fit the real scenario and make the test result more suitable for the real scenario.

[0078] Optionally, the adversarial sample generation algorithm includes a white-box test-based adversarial sample generation algorithm. The white-box test refers to a test in which an attacker can obtain the internal structure, training parameters, defense methods, and the like of the model.

[0079] Optionally, the adversarial sample generation algorithm currently used to test the intelligent recognition model can be replaced by other types of adversarial attack methods, such as a model stealing attack method.

[0080] Optionally, the original sample is a speech sample. Correspondingly, the adversarial sample generation algorithm includes, but is not limited to, CommanderSong (a demon music attack algorithm), a white-box adversarial speech generation method based on a principle of psychoacoustic concealment and a principle of iterative optimization, and a black-box adversarial speech generation method based on a genetic algorithm and gradient estimation.

[0081] In an example embodiment, the identification result is subjected to interference information generation processing corresponding to the interference information generation parameter and having a target interference intensity to obtain identification interference information in an adversarial attack scenario. The target interference intensity is determined according to an interference intensity parameter in the sample generation parameter. The interference intensity parameter includes a minimum intensity and a maximum intensity of adversarial interference, where the minimum intensity is not less than 1, and the maximum intensity is not greater than 32, i.e., 1≤ ≤ ≤32. An excessively large or small interference intensity is meaningless for a test result.

[0082] For an original sample , adversarial samples of the original sample x are generated according to the l adversarial perturbation modes selected in the above process, the i different intelligent identification models, the j adversarial sample generation algorithms, and the minimum intensity and the maximum intensity of adversarial interference in sequence. For an original sample, K adversarial samples can be generated, where K=i*j*l*( - +1). For all N original samples, the above process is repeatedly performed until the adversarial samples of the N original samples are generated. This process can generate N*K adversarial samples in total, effectively expands the coverage of the adversarial samples, improves the base of the test samples, and ensures the comprehensiveness and reliability of the test results.

[0083] In an example embodiment, the original sample is a speech sample. Therefore, when the above target identification processing is tested, the loss caused by the propagation of the speech signal in the environment needs to be considered. Correspondingly, as shown in Figure 4 , the implementation process of the step 213 further includes the following steps (2134-2135).

[0084] In the case where the adversarial perturbation mode includes a second adversarial perturbation mode, at least one speech propagation loss information is obtained.

[0085] The second adversarial perturbation mode refers to a test mode representing a propagation loss scenario.

[0086] The voice propagation loss information refers to signal loss information generated by voice signals propagating in an environment. Optionally, the voice propagation loss information includes at least one voice noise information. The at least one voice noise information includes, but is not limited to, environmental noise information in various environments, noise information caused by a propagation medium, and the like.

[0087] The voice propagation loss information can be preset voice propagation loss information, or voice propagation loss information corresponding to a current environment determined based on the current environment.

[0088] In step 2135, the voice propagation loss information is used as identification interference information in a propagation loss scenario.

[0089] The embodiments of the present application consider the attack effect of general adversarial samples, and include voice propagation loss information in a propagation loss scenario in the test range, which can effectively test the influence of normal noise samples on the robustness of the target identification processing, and improve the comprehensiveness of the test.

[0090] In step 214, at least one adversarial sample corresponding to the original sample is generated based on the original sample and the identification interference information in at least one interference scenario.

[0091] The identification interference information in at least one interference scenario is superimposed on the original sample to obtain an adversarial sample corresponding to the original sample.

[0092] In an example embodiment, the adversarial perturbation mode includes a first adversarial perturbation mode. Accordingly, as shown in Figure 4 The implementation process of step 214 includes step 2141 as shown below.

[0093] In step 2141, the identification interference information in an adversarial attack scenario is superimposed on the original sample to obtain an adversarial sample of the original sample in the adversarial attack scenario.

[0094] In a possible implementation, the identification interference information in the adversarial attack scenario includes pixel interference information, and the pixel interference information is superimposed on an image sample or a video sample to modify the pixel value corresponding to the image sample or the video sample to obtain an adversarial sample.

[0095] In an example embodiment, the adversarial perturbation mode includes a second adversarial perturbation mode. Accordingly, as shown in Figure 4 The implementation process of step 214 includes step 2142 as shown below.

[0096] At step 2142, the identification interference information in the propagation loss scenario is superimposed on the original sample to obtain an adversarial sample of the original sample in the propagation loss scenario.

[0097] In a possible implementation, the identification interference information in the propagation loss scenario includes noise information, and the noise information is superimposed on the speech sample to modify the speech data to obtain an adversarial sample.

[0098] The embodiments of the present application superimpose the identification interference information in the propagation loss scenario on the original sample, simulate the data quality loss in the actual application environment, obtain an adversarial sample of the original sample in the propagation loss scenario, and make the interference information carried by the adversarial sample more rich and comprehensive, thereby ensuring the completeness and accuracy of the test results.

[0099] In an example embodiment, the adversarial disturbance mode includes a third adversarial disturbance mode. The third adversarial disturbance mode refers to a test mode representing a composite interference scenario, and the composite interference scenario refers to an identification scenario with both propagation loss and adversarial attack. Correspondingly, as shown in FIG. 14, the implementation process of step 214 further includes the following steps (2143-2144). Figure 4

[0100] At step 2143, in the case where the adversarial disturbance mode includes the third adversarial disturbance mode, the identification interference information in the adversarial attack scenario is superimposed on the original sample to obtain an initial adversarial sample.

[0101] In a possible implementation, the identification interference information in the adversarial attack scenario can be superimposed on the original sample to obtain an initial adversarial sample. The initial adversarial sample can be understood as an adversarial sample corresponding to the original sample in the adversarial attack scenario.

[0102] The identification interference information in the adversarial attack scenario includes a plurality of identification interference information generated in the adversarial attack scenario.

[0103] At step 2144, the identification interference information in the propagation loss scenario is superimposed on the initial adversarial sample to obtain an adversarial sample of the original sample in the composite interference scenario.

[0104] Optionally, at least one noise information in the propagation loss scenario is superimposed on the initial adversarial sample to obtain an adversarial sample of the original sample in the composite interference scenario. The adversarial sample in the composite interference scenario simultaneously carries identification interference information in both the propagation loss and the adversarial attack scenarios.

[0105] ​For a real voice service scenario, the generation of the voice type of the adversarial sample needs to consider the auditory imperceptibility and environmental noise authenticity at the same time, the use of the adversarial sample for robustness testing needs to consider the propagation loss of the sound in the air, and unlike discrete image features, the voice signal features themselves have time domain continuity and complexity, and the robustness of the voice model should have the ability to resist background noise in the communication environment. Therefore, by using the adversarial sample in the above composite interference scene to test the above target recognition processing, the robustness of the target recognition processing to malicious adversarial attacks and signal propagation loss can be comprehensively evaluated, and the accuracy of the test result can be improved.

[0106] The generation process of the adversarial sample related to the embodiments of the present application determines the original sample needed for testing from the sample database through the sample data set parameter, generates at least one adversarial sample of the original sample in various interference scenes according to the adversarial disturbance mode determined by the adversarial disturbance mode parameter, the intelligent recognition model determined by the recognition processing parameter, the adversarial sample generation algorithm determined by the interference information generation parameter, and the interference intensity interval determined by the interference intensity parameter, expands the coverage of the adversarial sample, and can more comprehensively test the robustness of the target recognition processing.

[0107] Step 220, performing target recognition processing on the test sample to obtain the recognition information corresponding to the original sample and the adversarial sample respectively.

[0108] In the example embodiment, as shown in Figure 3 The step 220 can be replaced by the following step 221.

[0109] Step 221, inputting the test sample into the target recognition model to perform target recognition processing, and outputting the recognition information corresponding to the original sample and the adversarial sample respectively.

[0110] The target recognition model is a machine learning model trained based on the training sample and used for target recognition processing. The target recognition model is an intelligent recognition model to be tested.

[0111] In the example embodiment, the recognition information includes probability information corresponding to each type of label of the adversarial sample.

[0112] Optionally, the probability information includes output distribution information obtained by inputting the original sample into the target recognition model. The output distribution information can be the probability distribution of the original sample on each type of label.

[0113] Optionally, the probability information includes output distribution information obtained by inputting the adversarial sample into the target recognition model. The output distribution information can be the probability distribution of the adversarial sample on each type of label.

[0114] In an example embodiment, the identification information includes identified text information.

[0115] Optionally, the identified text information includes identified transcription text corresponding to the original sample after the target identification processing. For example, the original sample is input into the target identification model to obtain the identified transcription text corresponding to the original sample.

[0116] Optionally, the identified text information includes identified transcription text corresponding to the adversarial sample after the target identification processing. For example, the adversarial sample is input into the target identification model to obtain the identified transcription text corresponding to the adversarial sample.

[0117] The attack prediction result corresponding to the target identification processing of the test sample can be determined through the probability distribution corresponding to each of the original sample and the adversarial sample. Optionally, the attack prediction result includes data indicators such as identification accuracy, misprediction rate (i.e., attack success rate), and prediction confidence.

[0118] The transcription text error rate indicator corresponding to the target identification processing of the adversarial sample can be determined through the identified transcription text corresponding to each of the original sample and the adversarial sample. Optionally, the transcription text error rate indicator includes data indicators such as a first text error rate, a second text error rate, a third text error rate, a first average text error rate, and an average transcription change rate.

[0119] For related descriptions and determination processes of the data indicators, reference can be made to the following description. The data indicators can be used to determine at least one test performance indicator.

[0120] Step 230: obtaining sample information corresponding to each of the original sample and the adversarial sample.

[0121] The sample information includes, but is not limited to, content information, label information of the original sample and the adversarial sample, and original text information corresponding to each of the original sample and the adversarial sample.

[0122] In a possible implementation, the original sample is a speech sample, and correspondingly, the adversarial sample is also a speech sample. The original text information corresponding to each of the original sample and the adversarial sample is real text information corresponding to the speech sample.

[0123] Step 240: determining at least one test performance indicator corresponding to the target identification processing based on the sample information and the identification information.

[0124] The at least one test performance indicator represents the identification accuracy and the identification stability of the target identification processing from at least one dimension.

[0125] By determining at least one test performance indicator, the identification accuracy and the identification stability of the target identification process can be measured, and the identification prediction performance of the target identification process for specific information can be fully reflected.

[0126] In an example embodiment, the implementation of step 240 includes the following steps (241-243).

[0127] Step 241, based on the sample information corresponding to the original sample and the adversarial sample respectively, determining sample difference information.

[0128] In a possible implementation, the similarity between the adversarial sample and the original sample is measured to obtain the sample similarity between the adversarial sample and the original sample, and the sample similarity is taken as the sample difference information. Optionally, the similarity includes structural similarity (SS).

[0129] In the case where the original sample and the adversarial sample are speech samples, the similarity between the adversarial sample and the original sample is measured, the sound wave information of the two samples is obtained by Fourier transform processing the adversarial sample and the original sample, such as the shape of the sound wave, the direction of the speech gradient is represented by 1 and 0, the spectral characteristics of the sample are obtained, and the structural similarity is represented by calculating the Hamming distance of the two audio. The smaller the Hamming distance, the higher the structural similarity, and vice versa.

[0130] In an example, represents the Hamming distance of the audio waveform of the normal sample and the adversarial sample . represents the average Hamming distance between the adversarial sample and the original sample. Optionally, the inverse of the average Hamming distance is taken as the structural similarity, that is, .

[0131] Step 242, based on the identification information corresponding to the original sample and the adversarial sample respectively, determining identification difference information.

[0132] In a possible implementation, the identification information corresponding to the adversarial sample includes the identification label corresponding to the adversarial sample. Optionally, the identification label is the identification label output by the target identification model after the target identification process. The actual type label corresponding to the original sample and the identification label corresponding to the adversarial sample are compared to determine the identification difference information.

[0133] In a case where the adversarial perturbation mode is the fourth adversarial perturbation mode, the fourth adversarial perturbation mode refers to a test mode representing a non-interference scene, and the adversarial sample in the non-interference scene is the same as the original sample, and the identification difference information is an identification accuracy of the target identification processing (or a target identification model) In an example, the identification accuracy is determined by the following formula (1).

[0134] (1)

[0135] wherein, represents an i-th original sample an output distribution obtained by inputting the original sample into a target identification model, represents an original sample corresponding to an actual type label, is a sample serial number identifier, is a sample number of the original sample.

[0136] In a case where the adversarial perturbation mode is a non-fourth adversarial perturbation mode, the identification difference information is a misprediction rate of the target identification processing (or a target identification model) In an example, the misprediction rate is determined by the following formula (2).

[0137] (2)

[0138] wherein, is a sample number of the adversarial sample, represents an adversarial sample an output distribution obtained by inputting the adversarial sample into a target identification model, represents an adversarial sample corresponding to an original sample corresponding to an actual type label, is a sample serial number identifier.

[0139] Step 243, determining a first test performance index corresponding to the target identification processing based on the sample difference information and the identification difference information.

[0140] In a possible implementation, the misprediction rate The first test performance index corresponding to the target recognition processing is obtained by multiplying the average Hamming distance between the original sample and the adversarial sample. The first test performance index is a data index representing the recognition accuracy and the recognition stability of the target recognition processing according to the sample difference information and the recognition difference information. Specifically, the first test performance index is a data index representing the recognition accuracy and the recognition stability of the target recognition processing according to the structured similarity and the misprediction rate.

[0141] In one example, the first test performance index may be determined by the following formula (3).

[0142] (3)

[0143] wherein, represents the average Hamming distance between the original sample and the adversarial sample.

[0144] The greater the Hamming distance between the two, the more obvious the adversarial perturbation of the target recognition processing (or the target recognition model), and the smaller the misprediction rate (also understood as the attack success rate), which means that the robustness of the target recognition processing (or the target recognition model) is better, and vice versa.

[0145] In an exemplary embodiment, the sample information includes original text information, and the recognition information includes recognized text information. The implementation process of step 240 further includes the following steps (244-245).

[0146] Step 244, based on the original text information and the recognized text information corresponding to the original sample and the adversarial sample respectively, determine the text difference information.

[0147] In one possible implementation, the text difference information includes a transcription text error rate. The transcription text error rate includes a first text error rate between the recognized transcription text corresponding to the original sample and the adversarial sample respectively after the target recognition processing, a second text error rate between the actual text of the adversarial sample and the recognized transcription text of the adversarial sample after the target recognition processing, and a third text error rate between the actual text of the original sample and the recognized transcription text of the original sample after the target recognition processing. The text error rate WER between the two texts can be determined by the following formula (4).

[0148] (4)

[0149] wherein, represents the number of inserted words, represents the number of replaced words,​​ denotes the number of deleted words, denotes the number of words in the normal word sequence.

[0150] In a specific application, for normal samples and adversarial samples, the normal samples can be used as a reference to calculate the number of inserted words, replaced words, deleted words, and the like of the adversarial samples to solve the first text error rate.

[0151] For the corresponding texts before and after the target recognition processing of the adversarial samples, the actual text before input is used as a reference to calculate the number of inserted words, replaced words, deleted words, and the like of the recognition transcription text corresponding to the adversarial samples after the target recognition processing to solve the second text error rate.

[0152] For example, a normal sample is a piece of speech, and the content is "I love China", but an attacker can inject an inaudible noise such as high-frequency audio or audible environmental noise into this piece of audio to generate an adversarial speech sample, so that the result of the transcription of this sample may be 'hello', which is the difference between the samples before and after transcription.

[0153] For the corresponding texts before and after the target recognition processing of the original samples, the actual text before input is used as a reference to calculate the number of inserted words, replaced words, deleted words, and the like of the recognition transcription text corresponding to the original samples after the target recognition processing to solve the third text error rate.

[0154] For the selected N original samples, N*K adversarial samples can be generated through the above generation steps of the adversarial samples.

[0155] In one possible implementation, the first text error rate between the N*K adversarial samples and the corresponding recognition transcription texts of the original samples after the target recognition processing is determined, and then the first average text error rate corresponding to each first text error rate can be determined. In one example, the first average text error rate is , and can be determined by the following formula (5).

[0156] (5)

[0157] wherein, denotes the calculation of the first text error rate between the i-th original sample and the corresponding adversarial sample each corresponding recognition transcription text.

[0158] In addition, for the N*K adversarial samples, the second text error rate between the actual text of the N*K adversarial samples and the recognition transcription text obtained after the target recognition processing of the adversarial samples is determined. In one example, the second text error rate is and can be determined by the following formula (6).

[0159] (6)

[0160] wherein, represents the actual text of the adversarial sample, represents the recognized transcription text obtained after the target recognition processing of the adversarial sample.

[0161] In order to measure the degree of change of the text error rate of the adversarial sample relative to the original sample, a third text error rate between the actual text of the N original samples and the recognized transcription text obtained after the target recognition processing of the original sample is also calculated. In one example, the third text error rate is and can be determined by the following formula (7).

[0162] (7)

[0163] wherein, represents the actual text of the original sample, represents the recognized transcription text obtained after the target recognition processing of the original sample.

[0164] For an adversarial sample, its corresponding original sample can be found, and the degree of change of the two can be represented by the following formula (8).

[0165] (8)

[0166] wherein, is a small enough number (which can be set by oneself) to ensure that the denominator is not zero.

[0167] Correspondingly, for the N*K adversarial samples, the average transcription change rate of the corresponding original samples can be determined. Alternatively, the average transcription change rate can be determined by the following formula (9).

[0168] (9)

[0169] wherein, represents the second text error rate corresponding to the i-th adversarial sample, represents the third text error rate corresponding to the original sample corresponding to the i-th adversarial sample.

[0170] Generally, when the text error rate change of the adversarial sample compared to the original sample is smaller, and the misprediction rate (i.e. the attack success rate) of the adversarial sample is higher, it means that the attack is more successful, and the model robustness is worse, and vice versa.

[0171] At step 245, a second test performance index corresponding to the target recognition processing is determined based on the text difference information and the recognition difference information.

[0172] In a possible implementation, the second test performance index is determined based on the average transcription change rate and the misprediction rate. and the average transcription change rate to obtain the second test performance index corresponding to the target recognition processing. The second test performance index is a data index that comprehensively represents the recognition accuracy and the recognition stability of the target recognition processing according to the text difference information and the recognition difference information. Specifically, the second test performance index is a data index that comprehensively represents the recognition accuracy and the recognition stability of the target recognition processing according to the average transcription change rate and the misprediction rate.

[0173] In an example, the second test performance index may be determined by the following formula (10).

[0174] (10)

[0175] The second test performance index is negatively correlated with the robustness of the target recognition processing (or the target recognition model).

[0176] Optionally, the second test performance index is a data index determined according to the first average text error rate , the average transcription change rate , and the misprediction rate , and can comprehensively represent the recognition accuracy and the recognition stability of the target recognition processing.

[0177] In an example embodiment, the sample information includes label information corresponding to the adversarial sample, and the label information corresponding to the adversarial sample is an actual type label of an original sample corresponding to the adversarial sample. The recognition information includes probability information corresponding to each type label of the adversarial sample. Accordingly, as shown in FIG. 2B, the implementation process of step 240 further includes steps 246-248. Figure 3

[0178] At step 246, probability information corresponding to the actual type label of the adversarial sample is determined from the probability information corresponding to each type label of the adversarial sample.

[0179] The probability information corresponding to each type label of the adversarial sample includes probability values of the adversarial sample output by the target recognition processing corresponding to each type label. The actual type label refers to the true label of the original sample corresponding to the adversarial sample.

[0180] ​In a possible implementation, the actual type label corresponding to the original sample corresponding to the adversarial sample is determined, and then a probability value of the adversarial sample corresponding to the actual type label is determined from probability values of the adversarial sample corresponding to each type label. The probability value of the adversarial sample corresponding to each type label can be identification information obtained after the adversarial sample is subjected to the target identification processing, or identification information output by inputting the adversarial sample into the target identification model for target identification processing. The probability value of the adversarial sample corresponding to the actual type label can be used as the probability information of the adversarial sample corresponding to the actual type label.

[0181] In step 247, the prediction confidence information is determined based on the probability information of the adversarial sample corresponding to the actual type label.

[0182] The probability information of the adversarial sample corresponding to the actual type label is a probability value of the adversarial sample corresponding to the actual type label determined by the target identification processing. The prediction confidence is used to represent the identification accuracy and the identification stability of the target identification processing at the same time.

[0183] In a possible implementation, the prediction confidence of the adversarial sample corresponding to the actual type label in the test sample is determined by the target identification processing (or the target identification model). The prediction confidence can be determined by formula (11).

[0184] (11)

[0185] wherein, N is the number of original samples, is the number of adversarial samples corresponding to each original sample, i is a sample serial number, represents the output distribution of the adversarial sample input into the target identification model, which is used to represent the probability information of the adversarial sample corresponding to the actual type label, represents the actual type label of the original sample corresponding to the adversarial sample, corresponding to the adversarial sample.

[0186] The prediction confidence is positively correlated with the robustness of the target identification processing (or the target identification model).

[0187] In an exemplary embodiment, the identification accuracy , the misprediction rate , and the prediction confidence are collectively used to form an attack prediction result. The attack prediction result is an identification result obtained by using the test sample to attack the target identification processing (or the target identification model).

[0188] Step 248, determining a third test performance index corresponding to the target recognition based on the prediction confidence information and the recognition difference information.

[0189] In a possible implementation, the third test performance index is determined based on the prediction confidence information, the misprediction rate and the recognition difference information.

[0190] In an example, the third test performance index is determined by the following formula (12).

[0191] (12)

[0192] Step 250, determining a test result corresponding to the target recognition processing based on the at least one test performance index.

[0193] In an example embodiment, the implementation of step 250 includes the following processes (251-253) as shown in the following. Figure 3

[0194] Step 251, obtaining weight information corresponding to the at least one test performance index.

[0195] In an example embodiment, the weight information includes a weight impact factor corresponding to each test performance index in the at least one test performance index. The weight impact factor is used to measure the influence degree of different test performance indexes on the robustness of the target recognition processing (or the target recognition model). Optionally, the weight impact factor is pre-set.

[0196] Step 252, determining a robustness score corresponding to the target recognition processing based on the weight information and the at least one test performance index.

[0197] The robustness score is used to represent the recognition accuracy and the recognition stability as a whole.

[0198] In an example embodiment, the robustness score corresponding to the target recognition processing is determined based on the first test performance index, the second test performance index, the third test performance index and the corresponding weight impact factor. The robustness score is positively correlated with the recognition accuracy and the recognition stability.

[0199] In an example, the robustness score is determined by the following formula (13).

[0200] (13)

[0201] wherein,​​​​​​​​ , , are the first test performance index , the second test performance index , the third test performance index corresponding weight influence factor.

[0202] The embodiments of the present application utilize the misprediction rate and prediction confidence in the attack prediction result, combine the structured similarity of the two kinds of samples themselves, and the text error rate generated by the speech recognition transcription text, and design the above-mentioned three test performance indexes which can represent the robustness of the target recognition processing (or target recognition model) from multiple angles, avoiding the problem that the single test label leads to unreliable test results.

[0203] In the case where the above-mentioned adversarial perturbation mode only includes the fourth adversarial perturbation mode, the robustness score of the target recognition processing is determined based on the preset weight influence factor, the above-mentioned prediction confidence and the above-mentioned recognition accuracy . In one example, the above-mentioned robustness score is , which can be determined by the following formula (14).

[0204] (14)

[0205] wherein, is the preset weight influence factor.

[0206] Step 253, taking the robustness score as the test result.

[0207] After determining the above-mentioned test result, the corresponding configuration can be performed according to the above-mentioned test result. The application scenarios of the embodiments of the present application include but are not limited to two scenarios of intelligent recognition model deployment self-test and intelligent recognition model service selection.

[0208] For the intelligent recognition model deployment self-test scenario, a robustness score threshold can be set; according to the robustness score and the robustness score threshold, the qualification degree of the intelligent recognition model is determined. The above-mentioned robustness score threshold can be simply understood as the passing line of the model robustness score. If , it indicates that the above-mentioned target recognition processing is qualified, otherwise it is unqualified. The above-mentioned is the above-mentioned robustness score threshold.

[0209] Herein, the application of the embodiments of the present application in the intelligent speech model deployment self-test scenario is briefly described taking the intelligent speech recognition task as an example. Before being deployed online, the intelligent speech model can test the robustness of the intelligent speech model by using the test method proposed in the embodiments of the present application, evaluate the performance of the normal samples in the real environment and the ability to resist complex environment and malicious attacks. Further, the intelligent speech model provider can improve the model availability and robustness according to the test results of the embodiments of the present application, such as the robustness score mentioned above. For example, if the intelligent speech service being tested has weak defense ability against the adversarial samples generated by various adversarial sample generation algorithms, the robustness of the model can be improved by adding an effective defense layer in the model.

[0210] For the intelligent recognition model service selection scenario, the intelligent recognition model whose robustness score meets the preset score condition can be selected to perform the corresponding information recognition service.

[0211] Herein, the application of the embodiments of the present application in the intelligent speech model service selection scenario is briefly described taking the intelligent speech recognition task as an example. For the user of the intelligent speech model, the test method proposed in the present application can be used to test and evaluate the robust performance of the model to be selected, compare from multiple dimensions, and select the intelligent speech model with the best performance and the highest cost performance according to the use demand.

[0212] In one example, as shown in Figure 5 The technical architecture diagram of a speech recognition model robustness evaluation platform is exemplarily shown. First, the application background of the speech recognition model robustness evaluation platform is described. The performance of the existing AI service-based acoustic model often depends on the speech-text transcription ability and the text-speech simulation ability, for example, real-time translation of a speaker's voice into text, or inputting a piece of text to simulate the tone and intonation of the speaker, which pays more attention to the high accuracy and high availability of the model. However, due to the emergence of adversarial samples containing non-robust features, AI models are vulnerable to attacks by malicious attackers. For example, an attacker can make the speech model output an incorrect recognition result by adding imperceptible background noise to the normal speech input. In short, adversarial attacks pose a serious challenge to the security of AI speech models deployed online. Imagine that an attacker plays a pre-set adversarial speech sample, causing a car in the process of autonomous driving to recognize "stop" as "go", which may cause incalculable losses. Therefore, when judging the performance of a speech model, not only its accuracy and availability for normal input need to be considered, but also its security robustness against adversarial sample attacks need to be considered to effectively defend against malicious attacks.

[0213] The speech recognition model robustness evaluation platform provided by the embodiments of the present application uses an adversarial attack technology to perform black-box testing and white-box testing on an AI-based intelligent speech model, performs performance and reverse evaluation on the speech model from multiple dimensions and multiple security indicators, analyzes the vulnerability and attack surface of the intelligent speech model, and promotes defense by attack, thereby helping to improve the security performance of intelligent speech services and enhance the robustness thereof.

[0214] To achieve the above technical effects, the speech recognition model robustness evaluation platform mainly includes two modules, an adversarial sample generation module and a robustness evaluation module, which are briefly introduced as follows.

[0215] The adversarial sample generation module is mainly used for generating test samples for evaluating the robustness of the target intelligent speech model. The test samples include four parts, namely normal speech input samples, samples with environmental noise caused by air propagation, adversarial samples generated by different adversarial speech generation algorithms and various open-source speech models, and adversarial samples generated by different adversarial speech generation algorithms and various open-source speech models and added with the above environmental noise. The adversarial sample generation module maintains a speech database with all labels, an intelligent speech model library and an adversarial speech sample generation algorithm library. The speech database contains open-source, multi-language speech samples with various characteristics, including but not limited to Mozilla Common Voice, Tatoeba, VOiCES Dataset, VoxCeleb and the like. The intelligent speech model library includes but is not limited to common speech models, such as Google Voice, DeepSpeech, Kaldi ASR and the like. The adversarial speech sample generation algorithm includes but is not limited to CommanderSong, a white-box adversarial speech generation method based on the principle of "psychoacoustic concealment" and iterative optimization, and a black-box adversarial speech generation method based on genetic algorithm and gradient estimation. The adversarial sample generation module can generate various normal samples, noise samples, adversarial samples and composite samples with both adversarial and noise added to evaluate the robustness of the intelligent speech model to be detected, thereby ensuring the sample coverage from the perspective of general adversarial samples and improving the integrity of the test system.

[0216] The robustness evaluation module evaluates the robustness of the intelligent speech model to be evaluated from multiple angles, and designs multiple different test performance indicators to evaluate the intelligent speech model. According to the recognition results of the intelligent speech model on normal speech samples, real scene samples with complex environmental noise through air propagation, adversarial speech samples generated by an adversarial sample generation algorithm, and adversarial speech samples in a real environment generated by the adversarial sample generation algorithm, the structured similarity between the samples, the attack prediction result, and the transcription text error rate are determined, and then the three test performance indicators are determined. Finally, the platform outputs the robustness score of the intelligent speech model to be tested. The intelligent speech model provider and user can determine the test results according to the embodiments of the present application before model deployment to evaluate the robustness of the intelligent speech service, which is used for model performance improvement and model selection.

[0217] In summary, the technical scheme provided by the embodiments of the present application can obtain the respective recognition information of the original sample and the adversarial sample by performing target identification processing on the original sample and the adversarial sample in the test sample. In order to make the test results of the target identification processing more accurate and avoid the influence of the sample difference between the two samples on the test results, the obtained two kinds of recognition information can be combined with the original sample information of the original sample and the adversarial sample to determine at least one test performance indicator that can represent the recognition accuracy and the recognition stability of the target identification processing, and is a test performance indicator in different dimensions, which effectively improves the accuracy of the test performance indicator. Finally, the robustness of the target identification processing is comprehensively measured in at least one dimension and at least one test performance indicator, which can obtain more accurate, reliable and comprehensive test results, solves the problem of low accuracy of test results caused by single test indicator, and improves the accuracy and scientificity of the test results.

[0218] The following is an apparatus embodiment of the present application, which can be used to execute the method embodiments of the present application. For details not disclosed in the apparatus embodiments of the present application, please refer to the method embodiments of the present application.

[0219] Please refer to Figure 6 which shows a block diagram of a test device provided by an embodiment of the present application. The device has the functions of implementing the above-mentioned test method, which can be implemented by hardware or corresponding software executed by hardware. The device can be a computer device or can be arranged in a computer device. The device 600 can include a sample acquisition module 610, a sample identification module 620, an information acquisition module 630, a test indicator determination module 640, and a test result determination module 650.

[0220] The sample obtaining module 610 is configured to obtain a test sample, the test sample including an original sample and an adversarial sample corresponding to the original sample.

[0221] The sample identifying module 620 is configured to perform target identification processing on the test sample to obtain identification information corresponding to the original sample and the adversarial sample respectively.

[0222] The information obtaining module 630 is configured to obtain sample information corresponding to the original sample and the adversarial sample respectively.

[0223] The test index determining module 640 is configured to determine at least one test performance index corresponding to the target identification processing based on the sample information and the identification information, the at least one test performance index representing the identification accuracy and the identification stability of the target identification processing from at least one dimension.

[0224] The test result determining module 650 is configured to determine a test result corresponding to the target identification processing based on the at least one test performance index.

[0225] In an example embodiment, the test index determining module 640 includes a sample difference determining unit, an identification difference determining unit, and a first index determining unit.

[0226] The sample difference determining unit is configured to determine sample difference information based on the sample information corresponding to the original sample and the adversarial sample respectively.

[0227] The identification difference determining unit is configured to determine identification difference information based on the identification information corresponding to the original sample and the adversarial sample respectively.

[0228] The first index determining unit is configured to determine a first test performance index corresponding to the target identification processing based on the sample difference information and the identification difference information.

[0229] In an example embodiment, the sample information includes original text information, the identification information includes identification text information, and the test index determining module 640 further includes a text difference determining unit and a second index determining unit.

[0230] The text difference determining unit is configured to determine text difference information based on the original text information and the identification text information corresponding to the original sample and the adversarial sample respectively.

[0231] The second index determining unit is configured to determine a second test performance index corresponding to the target identification processing based on the text difference information and the identification difference information.

[0232] In an example embodiment, the test result determination module 650 comprises a weight obtaining unit, a robustness score determination unit, and a test result determination unit.

[0233] The weight obtaining unit is configured to obtain weight information corresponding to the at least one test performance indicator.

[0234] The robustness score determination unit is configured to determine a robustness score corresponding to the target recognition processing based on the weight information and the at least one test performance indicator, where the robustness score is used to represent the recognition accuracy and the recognition stability as a whole.

[0235] The test result determination unit is configured to take the robustness score as the test result.

[0236] In an example embodiment, the sample obtaining module 610 comprises a parameter obtaining unit, an original sample obtaining unit, an interference information generation unit, and an adversarial sample generation unit.

[0237] The parameter obtaining unit is configured to obtain sample generation parameters.

[0238] The original sample obtaining unit is configured to obtain the original sample.

[0239] The interference information generation unit is configured to generate recognition interference information in at least one interference scenario according to the sample generation parameters.

[0240] The adversarial sample generation unit is configured to generate at least one adversarial sample corresponding to the original sample based on the original sample and the recognition interference information in the at least one interference scenario.

[0241] In an example embodiment, the sample generation parameters comprise at least one of an adversarial perturbation mode parameter, a recognition processing parameter, and an interference information generation parameter, and the interference information generation unit comprises a mode determination subunit, a result obtaining subunit, and an interference information generation subunit.

[0242] The mode determination subunit is configured to determine an adversarial perturbation mode based on the adversarial perturbation mode parameter.

[0243] The result obtaining subunit is configured to obtain a recognition result of the original sample after the recognition processing corresponding to the recognition processing parameter, in a case where the adversarial perturbation mode comprises a first adversarial perturbation mode, where the first adversarial perturbation mode represents a test mode of an adversarial attack scenario.

[0244] The interference information generation subunit is configured to perform interference information generation processing corresponding to the interference information generation parameter on the recognition result and the original sample to obtain recognition interference information in the adversarial attack scenario.

[0245] In an example embodiment, the original sample is a speech sample, and the interference information generation unit further comprises a loss information acquisition subunit and an interference information determination subunit.

[0246] The loss information acquisition subunit is configured to, in a case where the adversarial disturbance mode comprises a second adversarial disturbance mode, acquire at least one piece of speech propagation loss information, the second adversarial disturbance mode being a test mode representing a propagation loss scenario.

[0247] The interference information determination subunit is configured to take the speech propagation loss information as the recognition interference information in the propagation loss scenario.

[0248] In an example embodiment, the adversarial sample generation unit comprises an adversarial interference superposition unit and a propagation loss superposition unit.

[0249] The adversarial interference superposition unit is configured to, in a case where the adversarial disturbance mode comprises a third adversarial disturbance mode, superimpose the recognition interference information in the adversarial attack scenario to the original sample to obtain an initial adversarial sample.

[0250] The propagation loss superposition unit is configured to superimpose the recognition interference information in the propagation loss scenario to the initial adversarial sample to obtain an adversarial sample of the original sample in a composite interference scenario.

[0251] The third adversarial disturbance mode represents a test mode representing the composite interference scenario, and the composite interference scenario represents a recognition scenario with both propagation loss and adversarial attack.

[0252] In an example embodiment, the sample recognition module 620 comprises a model recognition unit.

[0253] The model recognition unit is configured to input the test sample into a target recognition model to perform the target recognition processing, and output recognition information corresponding to the original sample and the adversarial sample, respectively. The target recognition model is a machine learning model trained based on training samples and used to perform the target recognition processing.

[0254] In summary, the technical scheme provided by the embodiments of the present application can obtain the respective identification information of the original sample and the adversarial sample by performing target identification processing on the original sample and the adversarial sample in the test sample. In order to make the test result of the target identification processing more accurate and avoid the influence of the sample difference between the two samples on the test result, the obtained two kinds of identification information can be combined with the respective original sample information of the original sample and the adversarial sample to determine at least one test performance indicator that can represent the identification accuracy and the identification stability of the target identification processing, and is a test performance indicator in different dimensions, which effectively improves the accuracy of the test performance indicator. Ultimately, the robustness of the target identification processing is comprehensively measured by at least one test performance indicator in at least one dimension, a more accurate, reliable and comprehensive test result can be obtained, the problem of low accuracy of the test result caused by single test indicator is solved, and the accuracy and scientificity of the test result are improved.

[0255] It should be noted that the device provided in the above embodiments is only exemplified by the division of the above functional modules when realizing its functions. In actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above described functions. In addition, the device and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be described here.

[0256] Please refer to Figure 7 which shows the structure block diagram of the computer device provided by an embodiment of the present application. The computer device can be a server or a terminal, and the above server or terminal is used to execute the above test method. Specifically:

[0257] The computer device 700 includes a central processing unit (CPU) 701, a system memory 704 including a random access memory (RAM) 702 and a read-only memory (ROM) 703, and a system bus 705 connecting the system memory 704 and the central processing unit 701. The computer device 700 also includes a basic input / output system (I / O system) 706 that helps transfer information between various devices in the computer, and a mass storage device 707 for storing an operating system 713, application programs 714 and other program modules 715.

[0258] The basic input / output system 706 includes the various components needed to display information, such as a display 708 and input devices 709, such as a mouse, keyboard, or electronic stylus, for inputting information. Both the display 708 and the input devices 709 are connected to the central processing unit 701 through an input / output controller 710 that is connected to the system bus 705. The basic input / output system 706 can also include the input / output controller 710 for receiving and processing input from a number of other devices, including a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 710 provides output to a display screen, printer, or other type of output device.

[0259] The mass storage device 707 is connected to the central processing unit 701 through a mass storage controller (not shown) connected to the system bus 705. The mass storage device 707 and its associated computer readable media provide nonvolatile storage for the computer device 700. That is, the mass storage device 707 can include a computer readable medium (not shown) such as a hard disk or a CD-ROM (Compact Disc Read-Only Memory) drive.

[0260] Without loss of generality, computer readable media can include computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), flash memory or other solid state memory technology, CD-ROM, digital video disc (DVD), or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. Computer storage media would not, however, include communication media including wired or wireless signaling media that communicate program code in a modulated data signal. The system memory 704 and mass storage device 707, mentioned above, can collectively be referred to as memory.

[0261] According to various embodiments of the present application, the computer device 700 can also operate in a networking environment via a network 712 to remote computers. That is, the computer device 700 can connect to the network 712 through a network interface unit 711 connected to the system bus 705, or can use the network interface unit 711 to connect to another type of network or remote computer system (not shown).

[0262] The memory also includes a computer program stored therein, and configured to be executed by one or more processors to implement the above-described test method.

[0263] In an example embodiment, a computer-readable storage medium is also provided, in which at least one instruction, at least one program, a code set or an instruction set is stored, and when executed by a processor, the at least one instruction, the at least one program, the code set or the instruction set implements the above-described test method.

[0264] Optionally, the computer-readable storage medium can include a ROM (Read Only Memory), a RAM (Random Access Memory), a SSD (Solid State Drives), an optical disc, etc. Among them, the random access memory can include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0265] In an example embodiment, a computer program product or a computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above-described test method.

[0266] It should be understood that "multiple" referred to herein refers to two or more. The "and / or" describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist together, and B exists alone. The character " / " generally represents that the associated objects before and after it are in an "or" relationship. In addition, the step numbers described herein only exemplarily show a possible execution order between steps, and in some other embodiments, the above steps can also be executed in a non-numbered order, such as two different numbered steps being executed simultaneously, or two different numbered steps being executed in an order opposite to the illustration, and the embodiments of the present application do not limit this.

[0267] The above only describes example embodiments of the present application and does not limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A test method characterized by, The method comprises: obtaining a test sample, the test sample comprising an original sample and at least one adversarial sample corresponding to the original sample; the at least one adversarial sample is generated based on recognition interference information in at least one interference scene of the original sample and according to sample generation parameters, the sample generation parameters comprising at least one of an adversarial perturbation mode parameter, a recognition processing parameter and an interference information generation parameter, and the generation mode of the recognition interference information in the at least one interference scene comprises: determining an adversarial perturbation mode based on the adversarial perturbation mode parameter; in the case where the adversarial perturbation mode comprises a first adversarial perturbation mode, obtaining a recognition result of the original sample after recognition processing corresponding to the recognition processing parameter, the first adversarial perturbation mode being a test mode representing an adversarial attack scene; and performing interference information generation processing corresponding to the interference information generation parameter on the recognition result and the original sample to obtain the recognition interference information in the adversarial attack scene; performing target recognition processing on the test sample to obtain recognition information corresponding to the original sample and the adversarial sample respectively; obtaining sample information corresponding to the original sample and the adversarial sample respectively; based on the sample information and the recognition information, determining at least one test performance index corresponding to the target recognition processing, the at least one test performance index representing the recognition accuracy and the recognition stability of the target recognition processing from at least one dimension; based on the at least one test performance index, determining a test result corresponding to the target recognition processing.

2. The method of claim 1, wherein, The method comprises: based on the sample information corresponding to the original sample and the adversarial sample respectively, determining sample difference information; based on the recognition information corresponding to the original sample and the adversarial sample respectively, determining recognition difference information; based on the sample difference information and the recognition difference information, determining a first test performance index corresponding to the target recognition processing.

3. The method of claim 2, wherein, The sample information comprises original text information, and the recognition information comprises recognition text information, and the method further comprises: based on the original text information and the recognition text information corresponding to the original sample and the adversarial sample respectively, determining text difference information; based on the text difference information and the recognition difference information, determining a second test performance index corresponding to the target recognition processing.

4. The method according to any one of claims 1 to 3, characterized in that, The method comprises: obtaining weight information corresponding to the at least one test performance index; based on the weight information and the at least one test performance index, determining a robustness score corresponding to the target recognition processing, the robustness score being used to represent the recognition accuracy and the recognition stability as a whole; taking the robustness score as the test result.

5. The method of claim 1, wherein, The original sample is a speech sample, and the generation manner of the recognition interference information in the at least one interference scene further includes: In a case where the adversarial perturbation mode includes a second adversarial perturbation mode, the second adversarial perturbation mode being a test mode representing a propagation loss scene, at least one speech propagation loss information is acquired; The speech propagation loss information is used as the recognition interference information in the propagation loss scene.

6. The method of claim 5, wherein, The generation of the at least one adversarial sample corresponding to the original sample based on the original sample and the recognition interference information in the at least one interference scene includes: In a case where the adversarial perturbation mode includes a third adversarial perturbation mode, the recognition interference information in the adversarial attack scene is superimposed on the original sample to obtain an initial adversarial sample; The recognition interference information in the propagation loss scene is superimposed on the initial adversarial sample to obtain an adversarial sample of the original sample in a composite interference scene; The third adversarial perturbation mode is a test mode representing the composite interference scene, and the composite interference scene is a recognition scene with both propagation loss and adversarial attack.

7. The method of claim 1, wherein, The target recognition processing of the test sample includes: The test sample is input into a target recognition model to perform the target recognition processing, and the recognition information corresponding to the original sample and the adversarial sample is output, the target recognition model being a machine learning model trained based on training samples and used for the target recognition processing.

8. A test device, characterized by The device includes: A sample acquisition module is configured to acquire a test sample, the test sample including an original sample and at least one adversarial sample corresponding to the original sample; the at least one adversarial sample being generated based on the original sample and recognition interference information in at least one interference scene obtained according to sample generation parameters, the sample generation parameters including at least one of an adversarial perturbation mode parameter, a recognition processing parameter, and an interference information generation parameter, and the generation manner of the recognition interference information in the at least one interference scene including: determining an adversarial perturbation mode based on the adversarial perturbation mode parameter; in a case where the adversarial perturbation mode includes a first adversarial perturbation mode, the first adversarial perturbation mode being a test mode representing an adversarial attack scene, acquiring a recognition result of the original sample after recognition processing corresponding to the recognition processing parameter; and performing interference information generation processing corresponding to the interference information generation parameter on the recognition result and the original sample to obtain recognition interference information in the adversarial attack scene; A sample recognition module is configured to perform target recognition processing on the test sample to obtain recognition information corresponding to the original sample and the adversarial sample; An information acquisition module is configured to acquire sample information corresponding to the original sample and the adversarial sample. The test index determination module is configured to determine at least one test performance index corresponding to the target recognition processing based on the sample information and the identification information, the at least one test performance index representing the identification accuracy and the identification stability of the target recognition processing from at least one dimension. The test result determination module is configured to determine a test result corresponding to the target recognition processing based on the at least one test performance index.

9. The apparatus of claim 8, wherein, The test index determination module includes: A sample difference determination unit configured to determine sample difference information based on the sample information corresponding to the original sample and the adversarial sample respectively. An identification difference determination unit configured to determine identification difference information based on the identification information corresponding to the original sample and the adversarial sample respectively. A first index determination unit configured to determine a first test performance index corresponding to the target recognition processing based on the sample difference information and the identification difference information.

10. The apparatus of claim 9, wherein, The sample information includes original text information, and the identification information includes identified text information. The test index determination module further includes: A text difference determination unit configured to determine text difference information based on the original text information and the identified text information corresponding to the original sample and the adversarial sample respectively. A second index determination unit configured to determine a second test performance index corresponding to the target recognition processing based on the text difference information and the identification difference information.

11. The apparatus of any one of claims 8 to 10, wherein, The test result determination module includes: A weight acquisition unit configured to acquire weight information corresponding to the at least one test performance index. A robustness score determination unit configured to determine a robustness score corresponding to the target recognition processing based on the weight information and the at least one test performance index, the robustness score representing the identification accuracy and the identification stability as a whole. A test result determination unit configured to take the robustness score as the test result.

12. The apparatus of claim 8, wherein, The original sample is a speech sample. The generation manner of the identification interference information in the at least one interference scene further includes: In a case where the adversarial perturbation mode includes a second adversarial perturbation mode, acquiring at least one speech propagation loss information, the second adversarial perturbation mode being a test mode representing a propagation loss scene. Taking the speech propagation loss information as the identification interference information in the propagation loss scene.

13. The apparatus of claim 12, wherein, The generation of the at least one adversarial sample corresponding to the original sample based on the original sample and the identification interference information in the at least one interference scene includes: In a case where the adversarial perturbation mode includes a third adversarial perturbation mode, superimposing the identification interference information in the adversarial attack scene to the original sample to obtain an initial adversarial sample. Superimposing the identification interference information in the propagation loss scene to the initial adversarial sample to obtain an adversarial sample of the original sample in a composite interference scene. The third adversarial perturbation mode represents the composite interference scene, and the composite interference scene represents an identification scene with both propagation loss and adversarial attack.

14. The apparatus of claim 8, wherein, The sample recognition module includes: The test sample is input into a target recognition model for the target recognition processing, and identification information corresponding to the original sample and the adversarial sample is output, the target recognition model being a machine learning model trained based on training samples and used for the target recognition processing.

15. A computer device, comprising: The computer device comprises a processor and a memory, and the memory stores at least one instruction, at least one program, a code set or an instruction set, which are loaded and executed by the processor to implement the test method according to any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, at least one program, a code set or an instruction set, which are loaded and executed by the processor to implement the test method according to any one of claims 1 to 7.

17. A computer program product, characterised in that, The computer program product comprises computer instructions stored in a computer readable storage medium, and the processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes to implement the test method according to any one of claims 1 to 7.

Images