Performance detection method of artificial intelligence model and related device

Abstract

The invention discloses a performance detection method of an artificial intelligence model and a related device, and relates to the technical field of artificial intelligence. Performance detection information is determined through a displayed creation task interface; obtaining a first link address, a second link address, a target sample data label and a target communication address based on the performance detection information; generating a test task based on the first link address, the second link address and the target sample data label; the server can obtain a target evaluation performance index based on the test task. Therefore, a capability management interface can be controlled and displayed, and the capability management interface is used for displaying the target evaluation performance index and the evaluation performance index corresponding to the target version number. Therefore, the purpose of determining whether the performance of the artificial intelligence model is remarkably improved or not is achieved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a performance detection method and related device for an artificial intelligence model. Background Art

[0002] As AI technology develops, AI models need to be continuously updated. Whether the updated AI model needs to be released depends on whether the performance is significantly improved. If the performance of the AI ​​model is significantly improved, it makes sense to re-release the AI ​​model because this allows users to benefit from these improvements.

[0003] The management of sample data and the performance of each version of the AI ​​model are complex and decentralized tasks. This decentralization makes it cumbersome to determine whether the performance of the AI ​​model has been significantly improved. Summary of the invention

[0004] In view of the above problems, the present application provides a performance detection method and related device for an artificial intelligence model to reduce the complexity of determining whether the performance of an artificial intelligence model has been significantly improved. The specific scheme is as follows:

[0005] The first aspect of the present application provides a performance detection method for an artificial intelligence model, comprising:

[0006] In response to the operation of creating a test task, controlling the display of a task creation interface;

[0007] Determine performance test information through the creation task interface, the performance test information including a target artificial intelligence model identifier, a target sample data identifier, a target test scenario, a target version number of the artificial intelligence model to be compared, and a target server identifier;

[0008] In response to determining to create a test task operation, searching for a first link address of the artificial intelligence model corresponding to the target artificial intelligence model identifier from a correspondence between a preset artificial intelligence model identifier and a link address of the artificial intelligence model;

[0009] Finding a second link address of the sample data corresponding to the target sample data identifier from a preset correspondence relationship between the sample data identifier and the link address of the sample data;

[0010] Finding the target sample data label corresponding to the target test scenario from the correspondence between the preset test scenarios and the sample data labels;

[0011] From the correspondence between the preset server identifier and the server's communication address, searching for the target communication address corresponding to the destination server identifier;

[0012] A test task is generated based on the first link address, the second link address and the target sample data label, wherein the test task is used to instruct the server with the target communication address to run the artificial intelligence model at the first link address, the input of the artificial intelligence model at the first link address being the sample data located at the second link address; and is used to instruct the server to obtain a target evaluation performance indicator based on the target sample data label and a prediction result; the prediction result is the output of the artificial intelligence model at the first link address;

[0013] Sending the test task to a server having the target communication address;

[0014] Obtaining the target evaluation performance indicator fed back by the server;

[0015] Find the evaluation performance indicator corresponding to the target version number from the preset correspondence relationship between the version number and the evaluation performance indicator;

[0016] Control and display a capability management interface, wherein the capability management interface is used to display the target evaluation performance indicator and the evaluation performance indicator corresponding to the target version number.

[0017] In a possible implementation, the performance detection information further includes a target evaluation script identifier, a target startup script template identifier, and script parameters, and the performance detection method of the artificial intelligence model further includes:

[0018] Based on the target evaluation script identifier, the target startup script template identifier and the script parameters, obtaining an instruction to generate an evaluation script;

[0019] The generate evaluation script instruction is used to instruct the server to fill the script parameters into the target startup script template to obtain a target evaluation script; the target startup script template is a template corresponding to the target evaluation script identifier and the target startup script template identifier found from the correspondence between the preset evaluation script identifier, the startup script template identifier and the startup script template;

[0020] Among them, the target evaluation script is used to obtain the target evaluation performance index based on the target sample data label and the prediction result.

[0021] In a possible implementation, the performance detection information further includes a target test environment identifier, and the performance detection method of the artificial intelligence model further includes:

[0022] The target interface number corresponding to the target test environment identifier is searched from the preset correspondence relationship between the test environment identifier and the interface number.

[0023] In a possible implementation, sending the test task to the server having the target communication address includes:

[0024] Sending the test task to the service corresponding to the target interface number contained in the server having the target communication address;

[0025] The server is used to run the artificial intelligence model at the first link address in the test environment corresponding to the target interface number.

[0026] In a possible implementation, the performance detection information also includes a target test type, wherein:

[0027] If the target test type is model perception, the test task is also used to instruct the server to obtain the target evaluation performance indicator of the artificial intelligence model at the first link address;

[0028] If the target test type is a capability test, the test task is further used to instruct the server to obtain the target evaluation performance index of the artificial intelligence model at the first link address and the performance index of the business logic script;

[0029] Among them, the output of the artificial intelligence model at the first link address is the input of the judgment business logic script, the judgment business logic script is used to evaluate whether there is a risk, and the judgment business logic script is stored in the server.

[0030] The second aspect of the present application provides a performance detection device for an artificial intelligence model, comprising:

[0031] A first control display module, used for controlling and displaying a task creation interface in response to a test task creation operation;

[0032] A first determination module is used to determine performance detection information through the creation task interface, wherein the performance detection information includes a target artificial intelligence model identifier, a target sample data identifier, a target test scenario, a target version number of the artificial intelligence model to be compared, and a target server identifier;

[0033] A first search module is used to search for a first link address of the artificial intelligence model corresponding to the target artificial intelligence model identifier from a correspondence relationship between a preset artificial intelligence model identifier and a link address of the artificial intelligence model in response to determining to create a test task operation;

[0034] A second search module, configured to search for a second link address of the sample data corresponding to the target sample data identifier from a preset correspondence relationship between the sample data identifier and the link address of the sample data;

[0035] A third search module, used to search for a target sample data label corresponding to the target test scenario from a preset correspondence relationship between the test scenario and the sample data label;

[0036] A fourth search module, used to search for a target communication address corresponding to the destination server identifier from a preset correspondence between the server identifier and the server's communication address;

[0037] A first generating module, for generating a test task based on the first link address, the second link address and the target sample data label, wherein the test task is used to instruct the server with the target communication address to run the artificial intelligence model at the first link address, the input of the artificial intelligence model at the first link address being the sample data located at the second link address; and for instructing the server to obtain a target evaluation performance indicator based on the target sample data label and a prediction result; the prediction result being the output of the artificial intelligence model at the first link address;

[0038] A first sending module, used for sending the test task to a server having the target communication address;

[0039] A first acquisition module, used to acquire the target evaluation performance indicator fed back by the server;

[0040] A fifth search module, used to search for the evaluation performance indicator corresponding to the target version number from the preset correspondence relationship between the version number and the evaluation performance indicator;

[0041] The second control display module is used to control the display of a capability management interface, where the capability management interface is used to display the target evaluation performance indicator and the evaluation performance indicator corresponding to the target version number.

[0042] In a possible implementation, the performance detection information further includes a target evaluation script identifier, a target startup script template identifier, and script parameters, and also includes:

[0043] A second acquisition module, configured to acquire an instruction for generating an evaluation script based on the target evaluation script identifier, the target startup script template identifier, and the script parameters;

[0044] The generate evaluation script instruction is used to instruct the server to fill the script parameters into the target startup script template to obtain a target evaluation script; the target startup script template is a template corresponding to the target evaluation script identifier and the target startup script template identifier found from the correspondence between the preset evaluation script identifier, the startup script template identifier and the startup script template;

[0045] Among them, the target evaluation script is used to obtain the target evaluation performance index based on the target sample data label and the prediction result.

[0046] The third aspect of the present application provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the performance detection method of the artificial intelligence model of the above-mentioned first aspect or any implementation method of the first aspect.

[0047] A fourth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0048] The memory is used to store computer programs;

[0049] The processor is used to execute the computer program so that the electronic device can implement the performance detection method of the artificial intelligence model of the above-mentioned first aspect or any implementation method of the first aspect.

[0050] In a fifth aspect, the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the performance detection method of the artificial intelligence model of the above-mentioned first aspect or any implementation method of the first aspect.

[0051] By means of the above technical scheme, the present application provides a performance detection method for an artificial intelligence model, which determines the performance detection information through a displayed creation task interface; from the correspondence between the preset artificial intelligence model identifier and the link address of the artificial intelligence model, find the first link address of the artificial intelligence model corresponding to the target artificial intelligence model identifier; from the correspondence between the preset sample data identifier and the link address of the sample data, find the second link address of the sample data corresponding to the target sample data identifier; from the correspondence between the preset test scenario and the sample data label, find the target sample data label corresponding to the target test scenario; from the correspondence between the preset server identifier and the communication address of the server, find the target communication address corresponding to the target server identifier; generate a test task based on the first link address, the second link address and the target sample data label; the server can obtain the target evaluation performance index based on the test task. Thus, the display capability management interface can be controlled, and the capability management interface is used to display the target evaluation performance index and the evaluation performance index corresponding to the target version number. Thus, the purpose of reducing the complexity of determining whether the performance of the artificial intelligence model has been significantly improved is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0053] Figure 1 A schematic diagram of a system architecture provided for this application;

[0054] Figure 2 A schematic diagram of an optional hardware structure of a terminal 100 provided in this application;

[0055] Figure 3 A schematic diagram of the structure of a server 200 provided in this application;

[0056] Figure 4 A schematic diagram of a performance testing method for an artificial intelligence model provided in this application;

[0057] Figure 5 A schematic diagram of an implementation method of creating a task interface provided by this application;

[0058] Figures 6a to 6c A schematic diagram of an interface for setting a correspondence between a sample data identifier and a link address of the sample data provided in an embodiment of the present application;

[0059] Figure 7 A schematic diagram of an implementation method of the capability management interface provided by this application;

[0060] Figure 8 A schematic diagram of an implementation of a window for displaying historical records provided by the present application;

[0061] Fig. 9 A schematic diagram of the structure of a performance detection device for an artificial intelligence model provided in this application;

[0062] Fig.10 A schematic diagram of the structure of the electronic device provided for this application; DETAILED DESCRIPTION

[0063] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0064] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0065] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0066] In the field of artificial intelligence, the management of sample data and the management of performance indicators for each updated version of artificial intelligence models is a complex and decentralized task. This decentralization leads to the tediousness of determining whether the performance of artificial intelligence models has been significantly improved. The following is an expanded description of this problem:

[0067] As the amount of sample data increases and the types of sample data diversify, effective sample data management becomes particularly important. Sample data needs to be cleaned, labeled, and maintained, and these tasks often require a lot of manpower and time.

[0068] AI models often need to be updated to adapt to new requirements. Each version of the AI ​​model needs to be tracked and compared to determine whether the new version of the AI ​​model is truly better than the old version of the AI ​​model. The lack of an effective version control mechanism makes it difficult to track the evolution and performance changes of the AI ​​model.

[0069] For example, when evaluating the performance of an artificial intelligence model (i.e., evaluating performance indicators), it is necessary to consider a variety of evaluation indicators, such as accuracy, precision, recall, F1 (F1 Score), ROC (Receiver Operating Characteristic) curve, and AUC (Area Under the Curve) value.

[0070] Based on this, the embodiment of the present application establishes a centralized management system for uniformly storing and managing the following information: sample data, artificial intelligence models of different version numbers, and evaluation performance indicators of artificial intelligence models of different version numbers. Such a centralized management system can record detailed information of each artificial intelligence model update, including changes in evaluation performance indicators, thereby simplifying the performance comparison process. The following is a detailed description.

[0071] See also Figure 1 , Figure 1A schematic diagram of a system architecture is shown. The system may include a terminal 100, a server 200 and a database 300. The server 200 may include one or more servers ( Figure 1 In the example, a server is included, and the server 200 can provide the method provided in the embodiment of the present application for one or more terminals.

[0072] Among them, an application or a web page can be installed on the terminal 100, and the above application and web page can provide an interface. The terminal 100 can receive relevant parameters entered by the user on the interface, such as performance detection information, and send the above parameters to the server 200. The server 200 can interact with the database 300 based on the received parameters to obtain processing results, and return the processing results to the terminal 100.

[0073] Exemplarily, terminal 100 is a terminal on the R&D side; database 300 may store the correspondence between the AI ​​model identifier and the link address of the AI ​​model, the correspondence between the sample data identifier and the link address of the sample data, the correspondence between the test scenario and the sample data label, and the correspondence between the server identifier and the communication address of the server. Exemplarily, database 300 may also store various AI models, various sample data, and various sample data labels.

[0074] Exemplarily, the number of databases 300 may be one or more.

[0075] It should be understood that in some optional implementations, the terminal 100 can also complete the action of interacting with the database 300 based on the received parameters to obtain the processing results by itself without the need for the cooperation of the server, and the embodiments of the present application are not limited thereto.

[0076] Next describe Figure 1 The product form of the mid-terminal 100;

[0077] The terminal 100 in the embodiment of the present application can be a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc., and the embodiment of the present application does not impose any limitation on this.

[0078] Figure 2 An optional hardware structure diagram of the terminal 100 is shown.

[0079] refer to Figure 2 As shown, the terminal 100 may include a radio frequency unit 110, a memory 120, an input unit 130, a display unit 140, a camera 150 (optional), an audio circuit 160 (optional), a speaker 161 (optional), a microphone 162 (optional), an earphone jack 163 (optional), a processor 170, an external interface 180, a power supply 190 and other components. Those skilled in the art will appreciate that Figure 2 This is merely an example of a terminal and does not constitute a limitation on the terminal. The terminal may include more or fewer components than shown in the figure, or a combination of certain components, or different components.

[0080] The input unit 130 can be used to receive input digital or character information, and generate key signal input related to the user settings and function control of the terminal. Specifically, the input unit 130 may include a touch screen 131 (optional) and / or other input devices 132. The touch screen 131 can collect the user's touch operations on or near it (such as the user's operation on or near the touch screen using any suitable object such as fingers, joints, stylus, etc.), and drive the corresponding connection device according to a pre-set program. The touch screen can detect the user's touch action on the touch screen, convert the touch action into a touch signal and send it to the processor 170, and can receive and execute the command sent by the processor 170; the touch signal at least includes the touch point coordinate information. The touch screen 131 can provide an input interface and an output interface between the terminal 100 and the user. In addition, the touch screen can be implemented using multiple types such as resistive, capacitive, infrared and surface acoustic wave. In addition to the touch screen 131, the input unit 130 can also include other input devices. Specifically, other input devices 132 may include, but are not limited to, one or more of a physical keyboard, function keys (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, and the like.

[0081] Among them, the input device 132 can receive input data and the like.

[0082] The display unit 140 may be used to display information input by the user or provided to the user, various menus of the terminal 100, interactive interfaces, file display and / or playback of any multimedia file. In the embodiment of the present application, the display unit 140 may be used to display interfaces, processing results, etc.

[0083] The memory 120 can be used to store instructions and data. The memory 120 can mainly include an instruction storage area and a data storage area. The data storage area can store various data, such as multimedia files, texts, etc.; the instruction storage area can store software units such as operating systems, applications, instructions required for at least one function, or their subsets and extensions. It can also include a non-volatile random access memory; provide the processor 170 with hardware, software and data resources including management of computing and processing equipment, and support control software and applications. It is also used for the storage of multimedia files, and the storage of running programs and applications.

[0084] The processor 170 is the control center of the terminal 100. It uses various interfaces and lines to connect various parts of the entire terminal 100. By running or executing instructions stored in the memory 120 and calling data stored in the memory 120, it executes various functions of the terminal 100 and processes data, thereby controlling the terminal device as a whole. Optionally, the processor 170 may include one or more processing units; preferably, the processor 170 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application program, and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 170. In some embodiments, the processor and the memory may be implemented on a single chip, and in some embodiments, they may also be implemented separately on separate chips. The processor 170 may also be used to generate corresponding operation control signals, send them to corresponding components of the computing and processing device, read and process data in the software, especially read and process data and programs in the memory 120, so that each functional module therein performs corresponding functions, thereby controlling the corresponding components to act according to the requirements of the instructions.

[0085] Among them, the memory 120 can be used to store software codes related to the performance detection method of the artificial intelligence model, the processor 170 can execute the steps of the performance detection method of the artificial intelligence model, and can also schedule other units (such as the above-mentioned input unit 130 and the display unit 140) to realize corresponding functions.

[0086] The radio frequency unit 110 (optional) can be used for receiving and sending information or receiving and sending signals during a call, for example, after receiving the downlink information of the base station, it is sent to the processor 170 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low Noise Amplifier, LNA), a duplexer, etc. In addition, the radio frequency unit 110 can also communicate with network devices and other devices through wireless communication. The wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile communication (Global System of Mobile communication, GSM), General Packet Radio Service (General Packet Radio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access, CDMA), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, WCDMA), Long Term Evolution (Long Term Evolution, LTE), email, Short Messaging Service (SMS), etc.

[0087] In this embodiment of the present application, the RF unit 110 can send data to the server 200 and receive processing results sent by the server 200.

[0088] It should be understood that the radio frequency unit 110 is optional and can be replaced by other communication interfaces, such as a network port.

[0089] The terminal 100 also includes a power supply 190 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 170 through a power management system, so that the power management system can manage functions such as charging, discharging, and power consumption.

[0090] The terminal 100 further includes an external interface 180 , which may be a standard Micro USB interface or a multi-pin connector, and may be used to connect the terminal 100 to communicate with other devices, or to connect a charger to charge the terminal 100 .

[0091] Although not shown, the terminal 100 may also include a flashlight, a wireless fidelity (WiFi) module, a Bluetooth module, sensors with different functions, etc., which are not described in detail here. Some or all of the methods described below may be applied in the following embodiments. Figure 2 In the terminal 100 shown.

[0092] Next describe Figure 1 The product form of the server 200;

[0093] Figure 3 A structural diagram of a server 200 is provided, such as Figure 3 As shown, the server 200 includes a bus 201, a processor 202, a communication interface 203, and a memory 204. The processor 202, the memory 204, and the communication interface 203 communicate with each other via the bus 201.

[0094] The bus 201 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0095] The processor 202 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0096] The memory 204 may include a volatile memory, such as a random access memory (RAM). The memory 204 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0097] Among them, the memory 204 can be used to store software codes related to the performance detection method of the artificial intelligence model, the processor 202 can execute the steps of the performance detection method of the artificial intelligence model of the chip, and can also schedule other units to implement corresponding functions.

[0098] It should be understood that the above-mentioned terminal 100 and server 200 can be centralized or distributed devices, and the processors in the above-mentioned terminal 100 and server 200 (such as processor 170 and processor 202) can be hardware circuits (such as application specific integrated circuit (ASIC), field-programmable gate array (FPGA), general-purpose processor, digital signal processor (DSP), microprocessor or microcontroller, etc.), or a combination of these hardware circuits. For example, the processor can be a hardware system with an instruction execution function, such as a CPU, DSP, etc., or a hardware system without an instruction execution function, such as an ASIC, FPGA, etc., or a combination of the above-mentioned hardware systems without an instruction execution function and hardware systems with an instruction execution function.

[0099] Reference Figure 4 , Figure 4 A flow chart of a performance detection method for an artificial intelligence model provided in an embodiment of the present application is as follows: Figure 4 As shown, a performance detection method for an artificial intelligence model provided in an embodiment of the present application may include steps S401 to S411, and these steps are described in detail below.

[0100] Step S401: In response to the operation of creating a test task, controlling the display of a task creation interface.

[0101] Exemplarily, the operation of creating a test task may be an operation of touching a corresponding key, or a voice operation, or a preset gesture operation.

[0102] Step S402: Determine performance detection information through the creation task interface, where the performance detection information includes a target artificial intelligence model identifier, a target sample data identifier, a target test scenario, a target version number of the artificial intelligence model to be compared, and a target server identifier.

[0103] In order for those skilled in the art to better understand the task creation interface provided by the embodiment of the present application, the following example is given to illustrate. Figure 5 , which is a schematic diagram of an implementation method of creating a task interface provided in an embodiment of the present application.

[0104] like Figure 5As shown, the target AI model identifier can be selected through the "Task Type" selection box. Exemplarily, the AI ​​model identifier is represented by the name of the AI ​​model. Exemplarily, the pre-set multiple AI model identifiers are: CV Intelligent Reasoning (Computer Vision Intelligent Reasoning), NLP (Natural Language Processing) large model reasoning, multimodal large model reasoning, and RAGAS (Retrieval-Augmented Generation Assessment) capability assessment.

[0105] CV intelligent reasoning refers to the process of analyzing, understanding, and reasoning about image or video data using artificial intelligence technologies such as deep learning in the field of computer vision. This includes tasks such as image recognition, target detection, semantic segmentation, and more complex scene understanding, such as behavior recognition and event prediction.

[0106] NLP large model reasoning refers to the process of analyzing and processing text data using large pre-trained models in the field of natural language processing. These models are usually pre-trained on large amounts of text data to learn general representations of language, and can then be fine-tuned on specific tasks. Tasks involved in NLP large model reasoning include text classification, sentiment analysis, question-answering systems, machine translation, etc. During the reasoning process, the model is able to understand the semantics of the text, generate responses, or perform other NLP tasks.

[0107] Multimodal large model reasoning refers to the process of reasoning with a large model that integrates information from multiple modalities (such as vision, text, audio, etc.). This kind of reasoning can process and understand the relationship between cross-modal data, for example, combining image content with related text descriptions, or matching speech signals with the speaker's facial expressions. Multimodal reasoning is particularly important in tasks such as visual question answering (VQA) and visual common sense reasoning (VCR), which requires the model to have cross-modal semantic understanding and common sense reasoning capabilities.

[0108] RAGAS capability evaluation is an evaluation framework for retrieval-augmented generation (RAG) systems. RAG systems combine the two steps of retrieval and generation, first retrieving relevant information from a large amount of data, and then generating answers or content based on this information. The RAGAS evaluation framework provides a set of indicators, such as context precision, context recall, faithfulness, and answer relevancy, for comprehensive evaluation of the performance of RAG systems. This evaluation method focuses not only on the quality of the final generated content, but also on how the system uses the retrieved information to improve the generated results.

[0109] In the embodiment of the present application, the selected “artificial intelligence model identifier” is referred to as the “target artificial intelligence model identifier”.

[0110] For example, it can be done by Figure 5 In the "Server ID" in the target server ID, select the target server ID. Exemplarily, the server ID can be represented by the name of the server. Exemplarily, different server IDs correspond to different types of servers. Exemplarily, different server IDs correspond to different test environments. Exemplarily, different test environments run in the same server.

[0111] For example, it can be done by Figure 5 Select the sample data identifier in "Data Selection". Exemplarily, the sample data identifier can be represented by the name of the sample data; Exemplarily, the type of the sample data can be: one or more of: voice, video, text or image.

[0112] For example, it can be done by Figure 5 Select the target test scenario in "Test scenario". It is understandable that a sample data may have multiple sample data labels, and the sample data labels of the same sample data in different test scenarios are not exactly the same. The following example is used to illustrate.

[0113] For example, the sample data is a sample image, and the sample image includes a person, an escalator, and a gas tank. The sample data labels of the sample image include the person's location area, the escalator's location area, and the gas tank's location area. If the test scene is an unmanned escalator, the sample data labels corresponding to the test scene are the person's location area and the escalator's location area; if the test scene is a safety hazard, the sample data labels corresponding to the test scene are the person's location area and the gas tank's location area.

[0114] In summary, the test scenario is used to indicate the type of sample data label. After the sample data is determined, the corresponding type of sample data label can be selected from the sample data labels corresponding to the sample data based on the determined test scenario. For example, the types of sample data labels indicated by the test scenario "unmanned escalator" include: people and escalators, so that "people's location area" and "escalator's location area" can be selected from the sample data labels "people's location area", "escalator's location area" and "gas tank's location area" corresponding to the sample data.

[0115] In summary, the results required to be predicted by the artificial intelligence model may be different in different test scenarios.

[0116] For example, it can be done by Figure 5In the "Compare Calibration Version" in the , select the target version number of the artificial intelligence model to be compared. For example, one or more target version numbers can be selected. The following is an example of the version number.

[0117] For the same artificial intelligence model AA, if the artificial intelligence model AA is trained on January 1, 2024, and its version number is determined to be 1.0; if the artificial intelligence model AA is updated on February 1, 2024, and the updated version number is determined to be 2.0; if the artificial intelligence model AA is updated again on March 1, 2024, and the updated version number is determined to be 3.0; if the updated artificial intelligence model AA needs to be evaluated again at the current time, the target version number of the artificial intelligence model to be compared can be selected from 1.0, 2.0 and 3.0. One or more.

[0118] Step S403: In response to determining to create a test task operation, the first link address of the artificial intelligence model corresponding to the target artificial intelligence model identifier is searched from the correspondence between the preset artificial intelligence model identifier and the link address of the artificial intelligence model.

[0119] Exemplarily, the operation of determining to create a test task may be an operation of touching a corresponding key, or a voice operation, or a preset gesture operation.

[0120] Exemplarily, the link address may be a storage address where the artificial intelligence model is stored in a database.

[0121] Step S404: searching for a second link address of the sample data corresponding to the target sample data identifier from the preset correspondence between the sample data identifier and the link address of the sample data.

[0122] Step S405: searching for a target sample data label corresponding to the target test scenario from the correspondence between preset test scenarios and sample data labels.

[0123] The following describes the process of setting the correspondence between the sample data identifier and the link address of the sample data.

[0124] like Figures 6a to 6c , which is a schematic diagram of an interface for setting the correspondence between a sample data identifier and a link address of the sample data provided in an embodiment of the present application.

[0125] For example, in response to the operation of creating a data set, the display may be controlled Figure 6a It is understandable that multiple sample data are needed to evaluate the artificial intelligence model. Then, multiple sample data used to evaluate the same artificial intelligence model can be stored in the same data set, that is, one data set includes one or more sample data; for example, Figure 5You can select the corresponding dataset identifier (which can be represented by the dataset name) through "Data Selection"; the sample data in the dataset with this dataset identifier are used to evaluate the artificial intelligence model.

[0126] pass Figure 6a You can set the name of the dataset you are creating. Assume that the name of the dataset you are creating is aas. After creating a dataset, you can enter the user interface corresponding to the dataset, such as Figure 6b shown.

[0127] pass Figure 6b You can enter the "Image Import" Figure 6c The user interface shown. Figure 6c The user interface shown can upload local images to the database 300. In this way, a corresponding relationship between sample data and the link address of the sample data, as well as a corresponding relationship between the sample data and the data set is established.

[0128] For example, "Text Import" is used to import text, and the imported text is used as sample text; "Video Import" is used to import video, and the imported video is used as sample video. "Text Import" and "Video Import" are not listed in the Figure 6b Shown in.

[0129] For example, by Figure 6b The user interface shown can select a sample image or sample text, and then input the sample data label of the sample data by touching the user interface displayed by “add label”, thereby establishing the corresponding relationship between the sample data and the sample data label and storing it in the database 300.

[0130] It is understandable that different data sets correspond to different test scenarios, and different test scenarios correspond to different sample data labels.

[0131] Step S406: searching for the target communication address corresponding to the destination server identifier from the preset correspondence between the server identifier and the server's communication address.

[0132] Exemplarily, the communication address includes but is not limited to an IP (Internet Protocol Address) address.

[0133] Step S407: Generate a test task based on the first link address, the second link address and the target sample data label.

[0134] The test task is used to instruct the server with the target communication address to run the artificial intelligence model at the first link address, wherein the input of the artificial intelligence model at the first link address is the sample data located at the second link address; and to instruct the server to obtain the target evaluation performance indicator based on the target sample data label and the prediction result; the prediction result is the output of the artificial intelligence model at the first link address.

[0135] Exemplarily, the artificial intelligence model at the first link address is an artificial intelligence model that has been trained or updated.

[0136] It is understandable that after the performance detection information is determined through the task creation interface, if the operation of determining to create a test task is performed, a test task can be generated.

[0137] Exemplarily, the target evaluation performance indicators include, but are not limited to: accuracy, precision, recall, F1 (F1 Score) score, ROC (Receiver Operating Characteristic) curve and AUC (Area Under the Curve) value, etc.

[0138] Assuming that the sample data is a sample image, the sample data label of the sample image includes the location area of ​​the person and the location area of ​​the escalator. After the sample data is input into the artificial intelligence model, the output prediction result of the artificial intelligence model includes the predicted location area of ​​the person and the predicted location area of ​​the escalator.

[0139] Step S408: Send the test task to the server having the target communication address.

[0140] Step S409: Obtain the target evaluation performance indicator fed back by the server.

[0141] Step S410: searching for the evaluation performance indicator corresponding to the target version number from the preset correspondence between the version number and the evaluation performance indicator.

[0142] Exemplarily, the “correspondence between version numbers and evaluation performance indicators” may be stored in the database 300 .

[0143] Step S411: controlling the display of a capability management interface, wherein the capability management interface is used to display the target evaluation performance indicator and the evaluation performance indicator corresponding to the target version number.

[0144] like Figure 7 The figure shows a schematic diagram of an implementation method of the capability management interface provided in an embodiment of the present application.

[0145] like Figure 7As shown, the capability management interface can display the following content: the test scenario selected through the task creation interface, the version number of the currently tested artificial intelligence model, and the test time.

[0146] Figure 7 The following takes obtaining target evaluation performance indicators including recall rate and precision rate as an example to illustrate.

[0147] For example, if you want to view the evaluation performance indicators of the AI ​​model of the target version number, you can trigger "History" to display Figure 8 The window shown.

[0148] like Figure 8 As shown in the figure, the evaluation performance indicators corresponding to the target version number selected through the task creation interface can be displayed. This allows you to check whether the performance of the artificial intelligence model has been significantly improved.

[0149] The embodiment of the present application provides a performance detection method for an artificial intelligence model, which determines the performance detection information through a displayed creation task interface; from the correspondence between the preset artificial intelligence model identifier and the link address of the artificial intelligence model, find the first link address of the artificial intelligence model corresponding to the target artificial intelligence model identifier; from the correspondence between the preset sample data identifier and the link address of the sample data, find the second link address of the sample data corresponding to the target sample data identifier; from the correspondence between the preset test scenario and the sample data label, find the target sample data label corresponding to the target test scenario; from the correspondence between the preset server identifier and the communication address of the server, find the target communication address corresponding to the target server identifier; generate a test task based on the first link address, the second link address and the target sample data label; the server can obtain the target evaluation performance index based on the test task. Thus, the display capability management interface can be controlled, and the capability management interface is used to display the target evaluation performance index and the evaluation performance index corresponding to the target version number. Thus, the purpose of reducing the complexity of determining whether the performance of the artificial intelligence model has been significantly improved is achieved.

[0150] In an optional implementation, the performance detection information also includes a target evaluation script identifier, a target startup script template identifier, and script parameters, and the performance detection method of the artificial intelligence model also includes the following steps: based on the target evaluation script identifier, the target startup script template identifier, and the script parameters, obtaining instructions for generating an evaluation script.

[0151] Among them, the generate evaluation script instruction is used to instruct the server to fill the script parameters into the target startup script template to obtain the target evaluation script; the target startup script template is a template corresponding to the target evaluation script identifier and the target startup script template identifier found from the correspondence between the preset evaluation script identifier, the startup script template identifier and the startup script template.

[0152] Among them, the target evaluation script is used to obtain the target evaluation performance index based on the target sample data label and the prediction result.

[0153] Combine the following Figure 5 The above contents are explained.

[0154] pass Figure 5 The "Job" in the example selects a target evaluation script ID from multiple evaluation script IDs. One evaluation script ID may include multiple startup script template IDs.

[0155] pass Figure 5 In the Startup Scripts section, select a target startup script template ID from multiple startup script template IDs. Figure 5 Enter the script parameters in "Please enter the startup script parameters".

[0156] Exemplarily, the “correspondence between the evaluation script identifier, the startup script template identifier and the startup script template” may be stored in the server.

[0157] In an optional implementation, the performance detection information also includes a target test environment identifier, and the performance detection method of the artificial intelligence model also includes: searching for the target interface number corresponding to the target test environment identifier from the correspondence between the preset test environment identifier and the interface number.

[0158] The server includes multiple interface numbers. Different interface numbers correspond to different services, and different test environments run in different services.

[0159] Exemplarily, the test environment includes, but is not limited to: DEV (Development Environment), FAT (Functional Acceptance Testing Environment), UAT (User Acceptance Testing Environment), and PROD (Production Environment).

[0160] Correspondingly, sending the test task to the server having the target communication address includes: sending the test task to the service corresponding to the target interface number included in the server having the target communication address.

[0161] The server is used to run the artificial intelligence model at the first link address in the test environment corresponding to the target interface number.

[0162] In an optional implementation, the performance detection information also includes a target test type, wherein if the target test type is model perception, the test task is used to instruct the server to obtain the target evaluation performance index of the artificial intelligence model at the first link address. If the target test type is a capability test, the test task is used to instruct the server to obtain the target evaluation performance index of the artificial intelligence model at the first link address and the performance index of the business logic script.

[0163] Among them, the output of the artificial intelligence model at the first link address is the input of the judgment business logic script, the judgment business logic script is used to evaluate whether there is a risk, and the judgment business logic script is stored in the server.

[0164] Exemplarily, the business logic script may be stored in a server, a database, or a terminal.

[0165] Combination Figure 5 , can be Figure 5 The "Test Type" shown selects a test type from model awareness, rule recognition, and ability testing.

[0166] Exemplarily, if the test type is model perception, it means that the artificial intelligence model has been updated, but the judgment business logic script has not been updated. At this time, the server needs to feedback the target evaluation performance indicators of the artificial intelligence model; if the test type is rule recognition, it means that the artificial intelligence model has not been updated, but the judgment business logic script has been updated. At this time, the server needs to feedback the performance indicators of the changed judgment business logic script. The performance indicators of the judgment business logic script may include accuracy. If the test type is capability test, it means that the artificial intelligence model has been updated and the judgment business logic script has been updated. At this time, the server needs to feedback the performance indicators of the changed judgment business logic script and the target evaluation performance indicators of the artificial intelligence model.

[0167] The following examples illustrate the artificial intelligence model and the business logic script.

[0168] Assume that the sample data is a sample image, and the sample data label of the sample image includes the location area of ​​the person and the location area of ​​the escalator. After the sample data is input into the artificial intelligence model, the artificial intelligence model can output the prediction result. The judgment business logic script can judge whether there is a risk in the process of riding the escalator based on the prediction result. It is understandable that with the development of technology, the judgment business logic script is also changing. For example, the safety rule determined in January 2024 is: the person's hand should hold the handle of the escalator. At this time, the judgment logic of the judgment business logic script is: judge whether the position of the person's hand area overlaps with the handle position area of ​​the escalator. If it overlaps, it means there is no risk. If it does not overlap, it means there is a risk. The safety rule after the change in May 2024 is: the person's hand should hold the handle of the escalator or the person's eyes should stare at the escalator. After the safety rules change, the judgment logic of the judgment business logic script is: judge whether the position of the person's hand area overlaps with the handle position area of ​​the escalator, or judge whether the person's line of sight is on the escalator; if both of the above are no, there is a risk, and if one of the above is yes, there is no risk.

[0169] The above introduces a performance detection method for an artificial intelligence model provided in an embodiment of the present application. The following will introduce a device for executing the performance detection method for the artificial intelligence model mentioned above.

[0170] See also Fig. 9 , Fig. 9 A schematic diagram of the structure of a performance detection device for an artificial intelligence model provided in an embodiment of the present application. Fig. 9 As shown, the performance detection device of the artificial intelligence model includes:

[0171] A first control and display module 901 is used to control and display a task creation interface in response to a test task creation operation;

[0172] A first determination module 902 is used to determine performance detection information through the creation task interface, wherein the performance detection information includes a target artificial intelligence model identifier, a target sample data identifier, a target test scenario, a target version number of the artificial intelligence model to be compared, and a target server identifier;

[0173] A first search module 903 is used to search for a first link address of an artificial intelligence model corresponding to the target artificial intelligence model identifier from a correspondence relationship between a preset artificial intelligence model identifier and a link address of an artificial intelligence model in response to determining the operation of creating a test task;

[0174] A second search module 904 is used to search for a second link address of the sample data corresponding to the target sample data identifier from a preset correspondence relationship between the sample data identifier and the link address of the sample data;

[0175] The third search module 905 is used to search for the target sample data label corresponding to the target test scenario from the correspondence between the preset test scenarios and the sample data labels;

[0176] The fourth search module 906 is used to search for the target communication address corresponding to the destination server identifier from the correspondence between the preset server identifier and the communication address of the server;

[0177] A first generating module 907 is used to generate a test task based on the first link address, the second link address and the target sample data label, wherein the test task is used to instruct the server with the target communication address to run the artificial intelligence model at the first link address, and the input of the artificial intelligence model at the first link address is the sample data located at the second link address; and to instruct the server to obtain a target evaluation performance indicator based on the target sample data label and the prediction result; the prediction result is the output of the artificial intelligence model at the first link address;

[0178] A first sending module 908, configured to send the test task to a server having the target communication address;

[0179] A first acquisition module 909 is used to acquire the target evaluation performance indicator fed back by the server;

[0180] A fifth search module 910 is used to search for the evaluation performance indicator corresponding to the target version number from the preset correspondence relationship between the version number and the evaluation performance indicator;

[0181] The second control and display module 911 is used to control and display a capability management interface, where the capability management interface is used to display the target evaluation performance indicator and the evaluation performance indicator corresponding to the target version number.

[0182] In an optional implementation, the performance detection information also includes a target evaluation script identifier, a target startup script template identifier, and script parameters, and also includes:

[0183] A second acquisition module, configured to acquire an instruction for generating an evaluation script based on the target evaluation script identifier, the target startup script template identifier, and the script parameters;

[0184] The generate evaluation script instruction is used to instruct the server to fill the script parameters into the target startup script template to obtain a target evaluation script; the target startup script template is a template corresponding to the target evaluation script identifier and the target startup script template identifier found from the correspondence between the preset evaluation script identifier, the startup script template identifier and the startup script template;

[0185] Among them, the target evaluation script is used to obtain the target evaluation performance index based on the target sample data label and the prediction result.

[0186] In an optional implementation, the performance detection information further includes a target test environment identifier, and further includes:

[0187] The sixth search module is used to search for the target interface number corresponding to the target test environment identifier from the preset correspondence relationship between the test environment identifier and the interface number.

[0188] In an optional implementation, the first sending module includes:

[0189] A sending unit, used for sending the test task to the service corresponding to the target interface number contained in the server having the target communication address;

[0190] The server is used to run the artificial intelligence model at the first link address in the test environment corresponding to the target interface number.

[0191] In an optional implementation, the performance detection information also includes a target test type, wherein:

[0192] If the target test type is model perception, the test task is also used to instruct the server to obtain the target evaluation performance indicator of the artificial intelligence model at the first link address;

[0193] If the target test type is a capability test, the test task is further used to instruct the server to obtain the target evaluation performance index of the artificial intelligence model at the first link address and the performance index of the business logic script;

[0194] Among them, the output of the artificial intelligence model at the first link address is the input of the judgment business logic script, the judgment business logic script is used to evaluate whether there is a risk, and the judgment business logic script is stored in the server.

[0195] The present application also provides an electronic device in an embodiment. Fig.10 As shown, it shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Fig.10 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0196] like Fig.10As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1008 to a random access memory (RAM) 1003. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 1003. The processing device 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0197] Typically, the following devices may be connected to the I / O interface 1005: an input device 1006 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1007 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1008 including, for example, a memory card, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Fig.10 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0198] Also provided in an embodiment of the present application is a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements a performance detection method for any artificial intelligence model provided in an embodiment of the present application.

[0199] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When one or more computer programs are executed by an electronic device, the electronic device can implement a performance detection method for any artificial intelligence model provided in an embodiment of the present application.

[0200] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0201] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0202] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0203] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A performance detection method for an artificial intelligence model, characterized in that: include: In response to the operation of creating a test task, controlling the display of a task creation interface; Determine performance test information through the creation task interface, the performance test information including a target artificial intelligence model identifier, a target sample data identifier, a target test scenario, a target version number of the artificial intelligence model to be compared, and a target server identifier; In response to determining to create a test task operation, searching for a first link address of the artificial intelligence model corresponding to the target artificial intelligence model identifier from a correspondence between a preset artificial intelligence model identifier and a link address of the artificial intelligence model; Finding a second link address of the sample data corresponding to the target sample data identifier from a preset correspondence relationship between the sample data identifier and the link address of the sample data; Finding the target sample data label corresponding to the target test scenario from the correspondence between the preset test scenarios and the sample data labels; From the correspondence between the preset server identifier and the server's communication address, searching for the target communication address corresponding to the destination server identifier; Generate a test task based on the first link address, the second link address and the target sample data label, wherein the test task is used to instruct the server with the target communication address to run the artificial intelligence model at the first link address, and the input of the artificial intelligence model at the first link address is the sample data located at the second link address; and used to instruct the server to obtain a target evaluation performance indicator based on the target sample data label and the prediction result; the prediction result is the output of the artificial intelligence model at the first link address; Sending the test task to a server having the target communication address; Obtaining the target evaluation performance indicator fed back by the server; Find the evaluation performance indicator corresponding to the target version number from the preset correspondence relationship between the version number and the evaluation performance indicator; Control and display a capability management interface, wherein the capability management interface is used to display the target evaluation performance indicator and the evaluation performance indicator corresponding to the target version number.

2. The performance detection method of the artificial intelligence model according to claim 1 is characterized in that: The performance detection information also includes a target evaluation script identifier, a target startup script template identifier, and script parameters. The performance detection method of the artificial intelligence model also includes: Based on the target evaluation script identifier, the target startup script template identifier and the script parameters, obtaining an instruction to generate an evaluation script; The generate evaluation script instruction is used to instruct the server to fill the script parameters into the target startup script template to obtain a target evaluation script; the target startup script template is a template corresponding to the target evaluation script identifier and the target startup script template identifier found from the correspondence between the preset evaluation script identifier, the startup script template identifier and the startup script template; Among them, the target evaluation script is used to obtain the target evaluation performance index based on the target sample data label and the prediction result.

3. The performance detection method of the artificial intelligence model according to claim 1 is characterized in that: The performance detection information also includes a target test environment identifier, and the performance detection method of the artificial intelligence model also includes: The target interface number corresponding to the target test environment identifier is searched from the preset correspondence relationship between the test environment identifier and the interface number.

4. The performance detection method of the artificial intelligence model according to claim 3 is characterized in that: The sending the test task to the server having the target communication address includes: Sending the test task to the service corresponding to the target interface number contained in the server having the target communication address; The server is used to run the artificial intelligence model at the first link address in the test environment corresponding to the target interface number.

5. The performance detection method of the artificial intelligence model according to claim 1 is characterized in that: The performance detection information also includes the target test type, wherein: If the target test type is model perception, the test task is further used to instruct the server to obtain the target evaluation performance indicator of the artificial intelligence model at the first link address; If the target test type is a capability test, the test task is further used to instruct the server to obtain the target evaluation performance index of the artificial intelligence model at the first link address and the performance index of the business logic script; Among them, the output of the artificial intelligence model at the first link address is the input of the judgment business logic script, the judgment business logic script is used to evaluate whether there is a risk, and the judgment business logic script is stored in the server.

6. A performance detection device for an artificial intelligence model, characterized in that: include: A first control display module, used for controlling and displaying a task creation interface in response to a test task creation operation; A first determination module is used to determine performance detection information through the creation task interface, wherein the performance detection information includes a target artificial intelligence model identifier, a target sample data identifier, a target test scenario, a target version number of the artificial intelligence model to be compared, and a target server identifier; A first search module is used to search for a first link address of the artificial intelligence model corresponding to the target artificial intelligence model identifier from a correspondence relationship between a preset artificial intelligence model identifier and a link address of the artificial intelligence model in response to determining to create a test task operation; A second search module, configured to search for a second link address of the sample data corresponding to the target sample data identifier from a preset correspondence relationship between the sample data identifier and the link address of the sample data; A third search module, used to search for a target sample data label corresponding to the target test scenario from a preset correspondence relationship between the test scenario and the sample data label; A fourth search module, used to search for a target communication address corresponding to the destination server identifier from a preset correspondence between the server identifier and the server's communication address; A first generating module, configured to generate a test task based on the first link address, the second link address and the target sample data label, wherein the test task is configured to instruct the server having the target communication address to run the artificial intelligence model at the first link address, wherein the input of the artificial intelligence model at the first link address is the sample data located at the second link address; and used to instruct the server to obtain a target evaluation performance indicator based on the target sample data label and the prediction result; the prediction result is the output of the artificial intelligence model at the first link address; A first sending module, used for sending the test task to a server having the target communication address; A first acquisition module, used to acquire the target evaluation performance indicator fed back by the server; A fifth search module, used to search for the evaluation performance indicator corresponding to the target version number from the preset correspondence relationship between the version number and the evaluation performance indicator; The second control display module is used to control the display of a capability management interface, where the capability management interface is used to display the target evaluation performance indicator and the evaluation performance indicator corresponding to the target version number.

7. The performance detection device of the artificial intelligence model according to claim 6, characterized in that: The performance detection information also includes a target evaluation script identifier, a target startup script template identifier, and script parameters, and also includes: A second acquisition module, configured to acquire an instruction for generating an evaluation script based on the target evaluation script identifier, the target startup script template identifier, and the script parameters; The generate evaluation script instruction is used to instruct the server to fill the script parameters into the target startup script template to obtain a target evaluation script; the target startup script template is a template corresponding to the target evaluation script identifier and the target startup script template identifier found from the correspondence between the preset evaluation script identifier, the startup script template identifier and the startup script template; Among them, the target evaluation script is used to obtain the target evaluation performance index based on the target sample data label and the prediction result.

8. A computer program product, characterized in that It includes computer-readable instructions, which, when executed on an electronic device, enable the electronic device to implement the performance detection method of the artificial intelligence model as described in any one of claims 1 to 5.

9. An electronic device, characterized in that: The method comprises at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the performance detection method of the artificial intelligence model as described in any one of claims 1 to 5.

10. A computer storage medium, characterized in that: The storage medium carries one or more computer programs, which, when executed by an electronic device, enable the electronic device to implement the performance detection method of the artificial intelligence model as described in any one of claims 1 to 5.

Images