Generation method and device of target model, electronic equipment and storage medium

Abstract

The invention discloses a target model generation method and device, electronic equipment and a storage medium, and relates to the field of artificial intelligence and big data, and the method comprises the steps: constructing an initial model, and carrying out the iterative training of the initial model; determining to-be-deployed target equipment of the target model, and converting the initial target model based on the target equipment and the quantization set to obtain a converted target model; and performing precision testing on the conversion target model based on the test set, comparing a first precision test result with a second precision test result, if the conversion target model does not have precision loss, storing the conversion target model as a final target model, and storing associated information corresponding to the target model. According to the method and the device, the technical problems of relatively low model development efficiency and easy generation of unknown precision loss during model deployment due to low degree of automation of split model construction due to focusing on some local links of model training or deployment in related technologies are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence and big data or other related technical fields, in particular, to a target model generation method and device, electronic equipment and storage medium. BACKGROUND

[0002] With the popularity of Internet of Things devices and the explosive growth of data volume, the integration of artificial intelligence (AI) and edge computing has become the focus of the industry. Edge computing aims to push computing power to the edge of the network to reduce dependence on data centers, reduce latency, and improve real-time data processing and security. AI algorithm models are the core of intelligent analysis, decision-making and prediction, and their application on the edge side requires higher performance, adaptability and management.

[0003] In related technologies, from algorithm model training development to deployment application involves multiple links, including dataset preparation, algorithm model training, target device model conversion, conversion model precision verification, algorithm model management and application deployment. Each link requires manual intervention, lacks end-to-end automated management processes, and has low model development efficiency, making it difficult to scale. In addition, in the precision verification part, attention is often focused on precision changes, while the distribution of confidence is ignored. Since the model has been quantized, the test indicators may not differ much, but the confidence will change, causing the precision indicators corresponding to the confidence to deviate, affecting the deployment effect of the model on heterogeneous devices, and leading to unknown precision loss during model deployment.

[0004] To address the above problems, no effective solutions have been proposed so far. SUMMARY

[0005] The embodiments of the present application provide a target model generation method and device, electronic equipment and storage medium, to at least solve the technical problems in related technologies that often focus on some local links of model training or deployment, which has low degree of automation in fragmented model construction, resulting in low model development efficiency and unknown precision loss during model deployment.

[0006] According to an aspect of some embodiments of the present application, a method for generating a target model is provided, including: parsing a generation request of a target model to obtain a project requirement, and constructing a sample data set for the target model to be generated based on the project requirement, and dividing the sample data set to obtain a training set, a test set and a quantization set, wherein the quantization set is used to adjust the quantization parameters of the model in the model quantization process; constructing an initial model based on the project requirement, and iteratively training the initial model using the training set to obtain a trained initial target model, performing precision testing on the initial target model using the test set to obtain a precision testing result, and generating a first precision testing result based on the precision testing result of the initial target model, wherein the precision testing result at least includes: average precision index, recall rate, confidence distribution information; determining a target device to which the target model is to be deployed, and transforming the initial target model based on the target device and the quantization set to obtain a transformed target model; performing precision testing on the transformed target model based on the test set to obtain a second precision testing result, and comparing the first precision testing result with the second precision testing result to obtain a comparison result; in the case that the comparison result indicates that there is no precision loss of the transformed target model relative to the initial target model, storing the transformed target model as a final target model, and storing associated information corresponding to the target model, wherein the stored target model is used for deploying an application, a corresponding transformed model is deployed for different to-be-deployed devices, and a confidence threshold of the model deployment model is set according to the associated information.

[0007] According to another aspect of the embodiments of the present application, a target model generation apparatus is also provided, comprising: a construction unit configured to analyze a target model generation request to obtain a project requirement, and construct a sample data set for a target model to be generated based on the project requirement, and divide the sample data set to obtain a training set, a test set and a quantization set, wherein the quantization set is used to adjust a quantization parameter of the model in a model quantization process; a training unit configured to construct an initial model based on the project requirement, and iteratively train the initial model using the training set to obtain a trained initial target model, perform precision testing on the initial target model using the test set to obtain a precision testing result, and generate a first precision testing result based on the precision testing result of the initial target model, wherein the precision testing result at least includes an average precision indicator, a recall rate and confidence distribution information; a conversion unit configured to determine a target device to which the target model is to be deployed, and convert the initial target model based on the target device and the quantization set to obtain a converted target model; a testing unit configured to perform precision testing on the converted target model based on the test set to obtain a second precision testing result, and compare the first precision testing result with the second precision testing result to obtain a comparison result; and a storage unit configured to store the converted target model as a final target model in a case where the comparison result indicates that there is no precision loss of the converted target model relative to the initial target model, and store associated information corresponding to the target model, wherein the stored target model is used for deployment application, a corresponding converted model is deployed for different to-be-deployed devices, and a confidence threshold of the deployed model is set according to the associated information.

[0008] According to another aspect of the embodiments of the present application, an electronic device is also provided, comprising one or more processors and a memory, the memory being configured to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any of the above target model generation methods.

[0009] In the present application, the generation request of the target model is parsed to obtain the project requirement, and a sample data set is constructed for the target model to be generated based on the project requirement, and the sample data set is divided to obtain a training set, a test set and a quantization set, wherein the quantization set is used to adjust the quantization parameter of the model in the model quantization process, and an initial model is constructed based on the project requirement, and the initial model is iteratively trained using the training set to obtain a trained initial target model, the initial target model is tested for accuracy using the test set to obtain an accuracy test result, and a first accuracy test result is generated based on the accuracy test result of the initial target model, wherein the accuracy test result at least includes: average accuracy index, recall rate, confidence distribution information, then the target device to be deployed for the target model is determined, and the initial target model is transformed based on the target device and the quantization set to obtain a transformed target model, and the transformed target model is tested for accuracy based on the test set to obtain a second accuracy test result, and the first accuracy test result and the second accuracy test result are compared to obtain a comparison result, and finally in the case that the comparison result indicates that there is no accuracy loss of the transformed target model relative to the initial target model, the transformed target model is stored as the final target model, and the associated information of the target model is stored, wherein the stored target model is used for deploying application, and corresponding transformed model is deployed for different to-be-deployed devices, and the confidence threshold of the deployed model is set according to the associated information.

[0010] In the present application, the data set for model training is constructed based on the project requirement, and the model is iteratively trained according to the project requirement, the model is transformed based on the target device to be deployed, and the accuracy test and model storage management after transformation are provided, which provides a whole-process generation scheme from training data construction, model training, model transformation to model accuracy test and model storage management, realizes end-to-end model generation, and improves the automation degree of model development. Through fine accuracy verification, the risk of unknown accuracy loss after model transformation is reduced, the reliability and stability of the model in actual deployment are enhanced, and thus the technical problem that in related technologies, some local links of model training or deployment are often focused on, the automation degree of such fragmented model construction is low, resulting in low model development efficiency, and unknown accuracy loss is prone to occur during model deployment. BRIEF DESCRIPTION OF DRAWINGS

[0011] The drawings described herein are used to provide further understanding of the present application, and form a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0012] Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing a generation method of a target model is shown;

[0013] Figure 2 is a flow chart of an optional target model generation method according to an embodiment of the present application;

[0014] Figure 3 is a flow chart of an optional target model generation method according to an embodiment of the present application;

[0015] Figure 4 is a flow chart of an optional target model generation method according to an embodiment of the present application;

[0016] Figure 5 is a flow chart of an optional target model generation method according to an embodiment of the present application;

[0017] Figure 6 is a flow chart of an optional target model generation method according to an embodiment of the present application;

[0018] Figure 7 is a flow chart of an optional target model generation method according to an embodiment of the present application;

[0019] Figure 8 is a flow chart of an optional target model generation method according to an embodiment of the present application;

[0020] Figure 9 is a flow chart of an optional target model generation method according to an embodiment of the present application; DETAILED DESCRIPTION

[0021] In order to make the personnel in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all the other embodiments obtained by the person of ordinary skill in the art without creative labor should belong to the scope of protection of the present application.

[0022] It should be noted that the terms "first", "second", and the like in the description and claims of the application and the above drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0023] It should be noted that the target model generation method and device in the present application can be used in the field of artificial intelligence and big data to construct a target model, and can also be used in any field other than the field of artificial intelligence and big data to construct a target model. The application field of the target model generation method and device in the present application is not limited.

[0024] It should be noted that the information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for display, analyzed data, etc.) collected in the present application are information and data authorized by the user or authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of related data comply with relevant laws, regulations and standards, necessary security measures are taken, do not violate public order and good customs, and provide corresponding operation portal for user to choose authorization or refusal. For example, the system and related users or institutions are provided with an interface to provide the user with a corresponding operation portal for the user to choose to agree or refuse the automatic decision result; if the user chooses to refuse, the expert decision process is entered.

[0025] The following embodiments of the present application can be applied to various target model generation systems / applications / devices. The present application develops the model from data management, model training, model conversion, precision verification to model storage management, reduces manual intervention and script execution in the middle process, improves the efficiency of algorithm development and deployment, and reduces the possibility of manual operation error. At the same time, through the standardized precision verification process system, the whole process operation of test set preparation, model reasoning and index calculation is standardized, and the performance evaluation results under different hardware platforms are ensured to be comparable and traceable. The information of the associated information of the model after training and conversion is stored, and the information of the model training, conversion and precision verification result is recorded, forming a traceable version, which is convenient for finding the related information of the model.

[0026] The present application will be described in detail below in conjunction with various embodiments.

[0027] Embodiment One

[0028] According to an embodiment of the present application, an embodiment of a method for generating a target model is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown.

[0029] The method embodiment provided by Embodiment One of the present application can be executed in a mobile terminal, a computer terminal or similar computing device. Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing the method for generating a target model is shown. As shown in Figure 1 , the computer terminal 10 (or mobile device) can include one or more processors 102 (the processor 102 can include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it can also include a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which can be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. Those skilled in the art can understand that Figure 1 The structure shown is only schematic, which does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 can include more or fewer components than those shown in Figure 1 , or have a different configuration than that shown in Figure 1 .

[0030] It should be noted that the one or more processors 102 and / or other data processing circuits described above can be referred to herein as "data processing circuits" in general. The data processing circuit can be embodied in whole or in part as software, hardware, firmware or any other combination. In addition, the data processing circuit can be a single independent processing module, or any one of the other elements combined into the computer terminal 10 (or mobile device) in whole or in part. As referred to in the embodiments of the present application, the data processing circuit serves as a processor control (for example, selection of a variable resistance terminal path connected to an interface).

[0031] The memory 104 can be used to store software programs of application software and modules, such as program instructions / data storage devices corresponding to the target model generation method in the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implements the target model generation method described above. The memory 104 can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory 104 can further include a memory remotely arranged with respect to the processor 102, which can be connected to the computer terminal 10 through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0032] The transmission device 106 is used to receive or send data via a network. Specific examples of the above-mentioned network can include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to be able to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, RF) module, which is used to communicate with the Internet in a wireless manner.

[0033] The display can be, for example, a touch screen type liquid crystal display (LCD), which can enable a user to interact with the user interface of the computer terminal 10 (or mobile device).

[0034] Under the above-mentioned operating environment, the present application provides a target model generation method as shown in Figure 2 The implementation subject of the method is a target model generation system.

[0035] Figure 2 is a flowchart of an optional target model generation method according to an embodiment of the present application, as shown in Figure 2 The method includes the following steps:

[0036] In the current AI ecosystem, especially in the fields of edge computing and Internet of Things, there are a large number of devices, from simple embedded cameras to high-performance edge servers, which carry out real-time data analysis and intelligent response tasks. However, the development and deployment of AI models are often limited by highly specialized technical barriers and cumbersome manual processes, resulting in low efficiency in the process from design to actual application of the model, and the performance of the model on different hardware platforms is often difficult to predict and guarantee.

[0037] The application scenarios of this application are mainly concentrated in enterprise-level projects that require rapid iteration and deployment of AI models, especially in the field of edge computing, such as smart cities, intelligent transportation, security monitoring, and intelligent manufacturing. In these scenarios, AI models must not only be trained in the cloud but also be efficiently deployed and run on a variety of hardware devices at the edge. This requires a complete solution that can span model training, conversion, accuracy verification, and management, achieving seamless integration of models and hardware.

[0038] The full-process automation method of this application significantly improves the development and deployment efficiency of AI models. Through the end-to-end automated process, the barriers between multiple links are eliminated, the number and complexity of manual intervention are reduced, and the iteration and adjustment of the model are faster. The standardized accuracy verification process can ensure that the performance and accuracy of the model remain consistent after being converted to different hardware platforms. This is especially important for models running on edge devices, because these devices often have limited resources, and the performance of the model directly affects the reliability of the entire system and user experience. By paying attention to the confidence distribution after the model conversion, this application can provide more refined accuracy verification and more accurate threshold recommendations for model deployment under different hardware conditions. This not only improves the efficiency of model deployment, but also ensures the optimal operating state of the model in various situations, thereby improving the performance and resource utilization of the overall system.

[0039] Step S201 , parse the target model generation request to obtain project requirements, build a sample data set for the target model to be generated based on the project requirements, and divide the sample data set into a training set, a test set, and a quantization set.

[0040] In step S201 above, the first task is to parse the target model generation request and extract the project requirements. These requirements may include the application scenarios, performance indicators, data types, and other required information of the algorithm model to be built. Based on the project requirements, corresponding sample data is prepared to construct a sample dataset. This dataset should be comprehensive and representative, reflecting the various situations the model will encounter. For example, for an object detection model, the sample dataset should include target images under various lighting conditions, angles, and occlusion conditions to ensure the model's generalization capabilities.

[0041] Secondly, the constructed sample data set is subdivided into three subsets: training set, test set and quantization set. The training set is used for model training, which is the basis for model learning rules; the test set is used for performance evaluation after model training, to ensure that the model has good generalization ability; the quantization set is used to adjust the quantization parameters of the model during model quantization. Through the quantization set, a balance point between model size, running speed and accuracy can be found. The use of the quantization set can help the model to maintain high accuracy while reducing memory occupation and computing resources, and improve the running efficiency on low-power devices.

[0042] In step S202, an initial model is constructed based on project requirements, and the initial model is iteratively trained using the training set to obtain a trained initial target model. The initial target model is tested for accuracy using the test set to obtain an accuracy test result, and a first accuracy test result is generated based on the accuracy test result of the initial target model.

[0043] In the above step S202, the system provides a model training function, and selects a suitable model architecture and algorithm type based on project requirements. For example, if the project requirement is real-time traffic sign recognition, a detection algorithm may be selected, and an initial model is constructed based on a deep learning model framework for real-time object detection.

[0044] After the initial model is constructed, the initial model is iteratively trained using the pre-constructed training set. The training set contains various instances that the model should learn to recognize. Through repeated learning and weight adjustment on the training set, the model gradually optimizes its performance to meet or exceed the predetermined recognition accuracy target. After each training cycle, the model needs to be evaluated on the test set, which is a data set independent of the training set, to measure the model's generalization ability and actual performance. Through accuracy testing on the test set, the accuracy of the model on unknown data can be obtained. This process is repeated until the initial model reaches a satisfactory state, and a trained initial target model is obtained.

[0045] Finally, the trained initial target model is tested for accuracy using the pre-constructed test set to obtain a first accuracy test result. The accuracy test result of the model includes at least the average accuracy indicator. For example: AUC (Area Under the Curve, ROC curve area), AP (Average Precision, average precision), recall rate and confidence distribution information. The generation of the first accuracy test result is one of the key points in the entire process, which not only reflects the current performance of the model on a given task, but also provides a benchmark for the performance comparison of the transformed model. By comparing the performance of the model before and after transformation on the same test set, the performance loss during model transformation can be accurately evaluated.

[0046] Step S203, determine the target device to which the target model is to be deployed, and transform the initial target model based on the target device and the quantization set to obtain a transformed target model.

[0047] In the above step S203, the embodiment of the present application provides a full-process scheme from data preparation to model generation. After the initial target model is constructed, it is transformed into a general format. However, in actual use, when the model is deployed on different hardware devices, based on multiple factors such as computing architecture, computing capability, memory limitation, power consumption requirement, etc., the initial target model in the general format needs to be transformed.

[0048] Firstly, the hardware platform or device to which the target model is to be deployed needs to be determined. The selection of the device is directly related to the model transformation strategy and the final performance. For example, if the target device is an edge computing server, it may be equipped with a GPU or NPU, which will guide the model to adopt a transformation strategy more suitable for GPU (Graphics Processing Unit) acceleration; if it is a resource-limited embedded device, such as a monitoring camera, the model needs to pay more attention to compression and optimization to adapt to lower computing capability and memory capacity.

[0049] Secondly, the initial target model and the pre-constructed quantization set are downloaded to the target device, and the initial target model is transformed. The main purpose of model transformation is to adjust the model structure and parameters to make them more suitable for the hardware characteristics of the target device, so as to improve the running speed and resource use efficiency of the model on the target device while ensuring the detection accuracy. Model transformation specifically includes: model quantization, which includes weight quantization and activation value adjustment. The floating-point weights in the initial target model are converted to fixed-point numbers to reduce the model size and improve the inference speed, but at the same time the quantization parameters need to be adjusted through testing on the quantization set to ensure that the accuracy does not decrease significantly. Format transformation, according to the hardware of the target device, the model format is adjusted to convert the model format from the general format to the model format suitable for the target device. For example, the NVIDIA GPU (Graphics Processing Unit) device converts the general model into a highly optimized TensorRT Engine (.engine) file through the TensorRT tool chain; the Huawei Ascend NPU converts it into its supported offline model (.om) format through the CANN (Compute Architecture for Neural Networks) tool chain. Many domestic AI chips also have their own exclusive model formats and transformation tools.

[0050] Further, the step of transforming the initial target model based on the target device and the quantization set includes: obtaining the device type of the target device, determining the transformed specific model format based on the device type; obtaining the hardware specification information of the target device, and determining the quantization precision of the target device based on the hardware specification information; and transforming the initial target model in a general format into a transformed target model conforming to the specific model format by using the quantization set and based on the model format and the quantization precision.

[0051] Specifically, when transforming the initial target model, the device type of the target device is first obtained, so that the specific model format to be transformed is determined according to the device type. This step is crucial because different hardware supports different model formats. By selecting the correct format, it can be ensured that the model can be successfully executed on the target device and fully utilize the hardware characteristics of the device.

[0052] Before model transformation, it is also necessary to have a deep understanding of the specific hardware specifications of the target device, such as processor model, memory size, floating point operation capability, etc. These information will directly affect the model transformation strategy and parameter setting, such as the selection of quantization precision. Quantization precision refers to the degree of conversion of model weights and activation values from floating point numbers to fixed point numbers during model transformation. The selection of quantization precision needs to be based on the hardware specification information of the target device, as well as the trade-off between model performance and efficiency. Higher quantization precision (such as using more bits) may retain more precision, but will also increase the computational burden; on the contrary, lower quantization precision can reduce the use of computing resources, but may sacrifice certain detection accuracy.

[0053] The initial target model in a general format will be transformed by using the quantization set and based on the previously determined model format and quantization precision. This process involves converting the weights and parameters of the model from floating point number format to fixed point number format suitable for hardware calculation, while using the quantization set to fine-tune the quantization process to ensure that the transformed model can effectively run on the target device while maintaining as high precision as possible.

[0054] Further, the step of converting the initial target model in the general format into the converted target model in the specific model format based on the model format and the quantization precision comprises: inputting the quantization set into the initial target model as input data, and collecting the dynamic range and the quantization parameter of each network layer in the initial target model based on the output result of the initial target model and the quantization precision, wherein the dynamic range at least includes a weight range and an activation value range, and the quantization parameter at least includes a scaling factor and a zero point; converting the weight of each network layer of the initial target model from a floating point number to an integer representation based on the scaling factor, the zero point and the weight range; quantizing the activation value of each network layer based on the activation value range in the forward propagation process of the initial model to obtain a quantized target model; and converting the quantized target model from the general format into the specific model format to obtain the converted target model.

[0055] It should be noted that model quantization is a key operation in the model conversion process. When deploying a model on edge devices, embedded systems, smartphones or Internet of Things devices, these devices usually have limited computing power and storage space. Model quantization can help reduce the size of the model, making it run smoothly on these devices. In scenarios that require real-time data processing, such as autonomous vehicles, real-time video monitoring or speech recognition services, model quantization can significantly speed up the inference process of the model, thereby responding more quickly to real-time data streams. In large-scale data processing scenarios, model quantization can improve the throughput of servers, as it reduces the computing resources required for each inference, allowing servers to process more requests more quickly.

[0056] Specifically, in the model quantization stage, the quantization set is first used as input data and input into the initial target model in the general format through forward propagation. The quantization set is a specially selected set of data to represent various input situations that the model may encounter in actual operation. When processing these input data, the model generates a series of output results. Based on these output results and the pre-set quantization precision, the dynamic range and the quantization parameter of each network layer in the model can be collected and analyzed. The dynamic range includes the weight range and the activation value range, which are used to determine the value interval of the model parameters in the quantization process. The weight range refers to the minimum and maximum values of the weight parameters in the model, while the activation value range refers to the minimum and maximum values of the output of the activation function in each layer during the forward propagation process. The quantization parameter mainly includes the scaling factor and the zero point, which are used to convert floating point numbers to integers during the quantization process while minimizing information loss. The scaling factor is used to adjust the range of floating point numbers to make them suitable for conversion to integers, and the zero point is used to determine the offset of the integer representation to ensure that the quantized values can be correctly mapped back to the original floating point numbers.

[0057] Based on the collected scaling factors, zero points, and weight ranges, the next task is to convert the weights of each network layer in the initial target model from floating-point numbers to integer representations. This process utilizes quantization parameters to map floating-point weights to the integer domain through scaling and shifting operations to meet the data format requirements of the target device. The purpose of weight quantization is to reduce model size and improve running speed, especially for resource-constrained hardware platforms.

[0058] During the forward propagation process, the activation values of each network layer are quantized based on the collected activation value ranges to obtain the quantized target model. This step also adjusts the activation value based on the preset quantization precision to make it representable in integer form, thereby improving the computational efficiency of the model on the target device. Activation value quantization is a key step to ensure that the model maintains sufficient accuracy after quantization on the hardware.

[0059] Finally, the quantized target model is converted from a general format to a specific model format to adapt to the inference engine and computing environment of the target device. This conversion involves converting the model from a general format to other specific model formats that better match the hardware characteristics of the target device, providing optimized running efficiency and resource management.

[0060] The initial target model in a general format is converted into a highly optimized converted target model to adapt to the hardware environment of a specific target device. This process not only considers the quantization of model parameters but also precisely matches the computing power and format requirements of the target device, ensuring that the model still maintains high accuracy after conversion while significantly improving running efficiency and resource utilization on the target device.

[0061] Step S204, based on the test set, the accuracy of the converted target model is tested, and the first accuracy test result and the second accuracy test result are compared to obtain the comparison result.

[0062] It should be noted that during the model conversion process, unknown accuracy loss of the model may occur based on various factors, and the confidence distribution of the prediction result may also change. Therefore, the converted model needs to be finely tested and checked for accuracy. The same test set as the model training is used to test the accuracy of the converted target model obtained after conversion, to obtain a second test result. During the accuracy test, different false detection rates or accuracies are set, and the recall rate and the corresponding confidence distribution are observed. Then, the first accuracy test result of the pre-conversion model is compared with the second accuracy test result of the post-conversion model to obtain a comparison result. It is determined whether the converted model has accuracy loss through the comparison result. If there is accuracy loss, return to step S203 to re-convert the initial target model. In actual deployment, the confidence threshold of the model is set based on the confidence of the converted model to avoid changes in the confidence distribution of the converted model.

[0063] Further, the step of testing the accuracy of the conversion target model based on the test set includes: inputting the test set as input data into the conversion target model, and collecting the average accuracy indicators of the conversion target model based on the output results of the conversion target model; drawing a first reference curve of the conversion target model, and sampling the points on the first reference curve to obtain the recall rate corresponding to the fixed false detection rate, wherein the first reference curve is a curve composed of false detection rate and recall rate; drawing a second reference curve of the conversion target model, and sampling the points on the second reference curve to obtain the recall rate corresponding to the fixed accuracy rate, wherein the second reference curve is a curve composed of accuracy rate and recall rate.

[0064] Specifically, during the accuracy test, the test set is input as input data into the conversion target model, and the average accuracy indicators are collected according to the output results of the conversion target model, including at least AUC (Area Under the Curve, area under the ROC curve) and AP (Average Precision, average accuracy rate). These indicators can intuitively reflect the performance of the model in the detection task.

[0065] Meanwhile, when evaluating the model performance, a fixed false positive rate or precision rate is set, and the corresponding recall rate value and confidence threshold distribution are observed. The false positive rate refers to the ratio of the model incorrectly determining negative samples as positive samples, and the lower the false positive rate, the higher the model's control of false positives. The precision rate is the proportion of true positive samples among all samples predicted by the model as positive samples, reflecting the model's prediction accuracy. The recall rate is the proportion of samples successfully predicted by the model as positive samples among all actual positive samples. The confidence threshold refers to the limit value used by the model to determine the reliability of the prediction result when making predictions. The confidence threshold provides a decision basis for the model's prediction results, helping to judge the reliability of the model's output. At the same time, by adjusting the confidence threshold, the balance between the model's false positive rate and precision rate and recall rate can be flexibly controlled.

[0066] Based on the accuracy performance of the model under different confidence thresholds, a first reference curve and a second reference curve are drawn. The first reference curve has the false positive rate as the horizontal coordinate and the recall rate as the vertical coordinate. By sampling the first reference curve, the corresponding recall rate and confidence threshold distribution under a fixed false positive rate can be obtained. The confidence threshold distribution indicates the recognition confidence level of the model for positive samples under a certain false positive rate. The second reference curve has the precision rate as the horizontal coordinate and the recall rate as the vertical coordinate. By sampling the second reference curve, the corresponding recall rate and confidence threshold distribution under a fixed precision rate can be obtained, further supplementing the multi-dimensional analysis of the model's performance. At the same time, based on the confidence distribution of the model, it is also helpful to determine the best threshold setting of the model in actual deployment.

[0067] Further, the step of comparing the first accuracy test result with the second accuracy test result to obtain a comparison result includes: comparing the average accuracy index of the first accuracy test result with the average accuracy index of the second accuracy test result to obtain a first comparison result; comparing the recall rate corresponding to the fixed false positive rate in the first accuracy test result with the recall rate corresponding to the fixed false positive rate in the second accuracy test result to obtain a second comparison result; comparing the recall rate corresponding to the fixed precision rate in the first accuracy test result with the recall rate corresponding to the fixed precision rate in the second accuracy test result to obtain a third comparison result; and obtaining the comparison result based on the first comparison result, the second comparison result, and the third comparison result.

[0068] By comparing the average precision indicators and confidence distribution of the model before and after transformation under the same test set, it can be determined whether the transformed target model has precision loss. Specifically, the average precision indicators in the "first precision test result" are directly compared with the corresponding indicators in the "second precision test result" to obtain the "first comparison result". The average precision indicators include key performance indicators such as false detection rate, precision, recall rate, etc., which reflect the overall performance of the model in processing the test set. By directly comparing these indicators, the trend of model transformation in terms of precision can be quickly evaluated, whether the precision is improved, stable or decreased. Next, attention is paid to the comparison of recall rate under the condition of fixed false detection rate, and then the comparison of recall rate under the condition of fixed precision. By comparing the recall rate distribution before and after transformation, the robustness of the model transformation process can be verified. This is crucial to ensure that the model can maintain stable and high-quality prediction under various hardware limitations when deployed across platforms. At the same time, during the model transformation process, the confidence threshold distribution may change due to hardware differences, data format conversion, etc. It is possible that the average precision, recall rate precision do not have precision loss, but the confidence changes, so when deploying the model, the test results of the transformed model should be used to set the confidence threshold of the deployed model to ensure the consistency of model training and deployment effect.

[0069] Further, in the case where the comparison result indicates that the transformed target model does not have precision loss relative to the initial target model, the step of storing the transformed target model as the final target model includes: in the case where the first comparison result indicates that the difference between the average precision indicators of the first precision test result and the second precision test result is within the precision indicator difference range, determining that the transformed target model passes the average precision indicator test; in the case where the second comparison result indicates that the difference between the recall rate corresponding to the fixed false detection rate in the first precision test result and the recall rate corresponding to the fixed false detection rate in the second precision test result is within the first preset range, and in the case where the third comparison result indicates that the difference between the recall rate corresponding to the fixed precision rate in the first precision test result and the recall rate corresponding to the fixed precision rate in the second precision test result is within the second preset range, determining that the transformed target model passes the precision test; in the case where the transformed target model passes the average precision indicator test and the recall rate test, determining that the transformed target model does not have precision loss relative to the initial target model, obtaining the target model, and storing the target model; or, in the case where the transformed target model does not pass the average precision indicator test and the recall rate test, re-transforming the initial target model.

[0070] Specifically, if the difference between the average precision indicators of the target model before and after transformation is within the preset precision indicator difference range, it indicates that the transformed target model passes the average precision indicator test. If the recall rates of the target model before and after transformation under the fixed false detection rate and fixed accuracy rate are within the preset precision indicator difference range, it indicates that the transformed target model passes the recall rate test. Only when the transformed target model passes both the average precision indicator test and the recall rate test under the fixed false detection rate and accuracy rate, the transformed target model does not have precision loss, and thus it is stored as the final target model. The stored target model should include all information after transformation, ensuring that the model can be accurately deployed on the target device to achieve the expected prediction performance. However, if the transformed target model fails to pass the average precision indicator test and the recall rate test under the fixed false detection rate and accuracy rate, it means that the model has a significant precision drop during the transformation process. At this time, it is necessary to return to the model transformation stage, adjust the transformation parameters, or adjust the training parameters to retrain the algorithm model, in order to meet the preset precision indicator requirements in the next transformation.

[0071] By constructing a multi-dimensional performance test system, not only the average precision indicator of the transformed model is considered, but also the factor of confidence distribution change after transformation is considered, greatly improving the evaluation standard of model transformation quality. This not only helps to ensure that the transformed target model can maintain the same precision level as the original model, but also makes the deployment of the model on different hardware platforms more reliable and efficient.

[0072] Step S205, in the case that the comparison result indicates that the transformed target model does not have precision loss relative to the initial target model, the transformed target model is stored as the final target model, and the associated information corresponding to the target model is stored.

[0073] In the embodiments of the present application, a whole-process model construction scheme from data preparation to model generation is provided. In the case that the transformed target model does not have precision loss after transformation, the transformed target model is stored as the final target model, and the associated information of the target model is recorded and stored. Standardized storage and management of the transformed target model are realized, ensuring that the model can be properly saved and efficiently deployed after successful transformation. At the same time, the key information in the model transformation process is recorded and stored, and a whole life cycle management system of the model is constructed.

[0074] Further, the step of storing the associated information corresponding to the target model comprises: defining the algorithm type of the target model; determining whether the target model and the historical model are transformed from the same initial target model to obtain a determination result, and configuring a version number for the target model based on the determination result; collecting precision test information, a test set and training logs of the target model; obtaining the associated information corresponding to the target model based on the algorithm type, the version number, the precision test information, the test set and the training logs of the target model, and storing the associated information.

[0075] Specifically, when storing the associated information corresponding to the target model, it is necessary to first determine the algorithm type of the target model, such as whether it is a pedestrian detection model, a flame monitoring model or a vehicle recognition model. This definition provides a basis for model classification, facilitating subsequent management and retrieval. Subsequently, the system checks whether the target model and other historical models in the database are transformed from the same initial target model, which is to maintain the historical continuity of the model version. If the determination result is "yes", it means that the target model and the historical model share the same source, and at this time, the target model will inherit the version number of the historical model, maintaining the continuity of the version sequence; if the determination result is "no", it indicates that the target model comes from a new training process or model version, and at this time, the system will assign a unique version number to this new target model, ensuring that each model version has its own identification, facilitating model management and tracking. On this basis, the precision test information of the target model is collected, including but not limited to average precision indicators, recall rates and confidence threshold distributions. These data come from the precision test link and provide key indicators of the performance of the target model. At the same time, the test set and the training logs are collected, the latter of which records in detail the parameter adjustment and optimization process during model training, which is an important clue to understand the evolution path of the model, forming a traceable data.

[0076] Finally, all the collected associated information of the target model, such as precision information, threshold distribution information, model training information and test set information, is summarized to form a complete associated information data package corresponding to the target model, providing data reference for model deployment. This data package not only contains the basic attributes of the target model, but also encapsulates rich information about its performance and training history. Subsequently, through a systematic storage mechanism, these associated information is stored in a designated database or file system, ensuring the security and accessibility of the information.

[0077] Through the above steps, the generation request of the target model is parsed, the project requirement is obtained, the sample data set is constructed for the to-be-generated target model based on the project requirement, and the sample data set is divided to obtain the training set, the test set and the quantization set, wherein the quantization set is used to adjust the quantization parameter of the model in the model quantization process, the initial model is constructed based on the project requirement, the initial model is iteratively trained by using the training set to obtain the trained initial target model, the initial target model is tested for accuracy by using the test set to obtain the accuracy test result, the first accuracy test result is generated based on the accuracy test result of the initial target model, wherein the accuracy test result at least includes: average accuracy index, recall rate and confidence distribution information, then the type of the device to be deployed by the target model is determined, the initial target model is transformed based on the target device and the quantization set to obtain the transformed target model, the transformed target model is tested for accuracy based on the test set to obtain the second accuracy test result, and the first accuracy test result is compared with the second accuracy test result to obtain the comparison result, finally in the case that the comparison result indicates that there is no accuracy loss of the transformed target model relative to the initial target model, the transformed target model is stored as the final target model, and the associated information corresponding to the target model is stored, wherein the stored target model is used for deploying the application, and the corresponding transformed model is deployed for different to-be-deployed devices, and according to the associated information, the confidence threshold of the model deployment model is set.

[0078] In the embodiment, the data set for model training is constructed based on the project requirement, the model is iteratively trained according to the project requirement, the model is transformed based on the target device to be deployed, and the accuracy test and model storage management after transformation are provided, which provides a whole-process generation scheme from training data construction, model training, model transformation to model accuracy test and model storage management, realizes end-to-end model generation, reduces the risk of unknown accuracy loss after model transformation through fine accuracy verification, enhances the reliability and stability of the model in actual deployment, and further solves the technical problem in the related art that the model training or deployment is often focused on some local links, the fragmentation of the model construction has low automation degree, which leads to low model development efficiency, and unknown accuracy loss is easily generated in model deployment.

[0079] The following will be described in detail in combination with another optional specific embodiment.

[0080] Figure 3 is a schematic diagram of an optional target model generation process according to an embodiment of the application, as Figure 3As shown, the overall scheme for generating the target model includes: dataset management, model training, model conversion and model accuracy verification, and model storage. First, the dataset is constructed according to the project requirements, and the training set, test set and quantization set are distinguished. Then the training set is selected for algorithm model training, and the test set is used to evaluate the model indicators, and finally the original algorithm model (corresponding to the above initial target model) is obtained. By selecting the type of target device and the quantization dataset, the model is converted into a model format that matches the hardware device to be deployed (corresponding to the above target device). The converted model is used for accuracy verification, and the same test set as in the model training stage is selected to check if there is a loss of accuracy. If there is a large loss of accuracy, reconvert the model, adjust the conversion parameters, or adjust the training parameters and retrain the model, otherwise directly archive the model and store it according to the algorithm type, device type and version.

[0081] Figure 4 is a schematic diagram of an optional training process of a target model according to an embodiment of the present application, as shown in Figure 4 When training the model, first determine the algorithm type of the target model, such as target classification, segmentation, detection, and then select the corresponding model training framework, such as yolov5, yolov11, etc. Then apply for computing resource, use K8s centralized management training cluster to schedule computing resource, allocate corresponding GPU resource according to the demand of training task, and issue training task for iterative training of algorithm model. Based on the training and test data, the model training and testing are performed, and after the algorithm model training is completed, the optimal model, i.e. the model with the highest accuracy, is selected. Then the model is converted into a general format (such as.onnx) for subsequent deployment hardware format conversion.

[0082] Figure 5 is a schematic diagram of an optional conversion process of a target model according to an embodiment of the present application, as shown in Figure 5 The general format model is converted to the corresponding target device, compatible with multiple heterogeneous chip architectures. First, call the general format model, select the corresponding target device (such as NPU, GPU, Ascend, ARM chip device, etc.), complete the parameter configuration, and then issue the initial target model in the general format to the target device for model conversion. If model quantization is required, the quantization dataset is also issued. After the model conversion is completed, the converted target model is returned to the delivery platform for subsequent model accuracy verification process.

[0083] Figure 6 is a schematic diagram of an optional accuracy verification process of a target model according to an embodiment of the present application, as shown in Figure 6As shown, the converted model needs to be precision-checked to prevent precision loss during the conversion process. The model and test data (corresponding to the above test set) need to be sent to the corresponding test equipment (the test equipment corresponds to the target equipment, which can include NPU, GPU, Ascend, ARM chips, etc.) to ensure that the test set is consistent with the training process and perform the accuracy test of the model. The test results are compared with the initial target model in the training phase to determine whether there is precision loss. If there is precision loss, return to the model conversion phase to modify the parameters and reconvert the model or adjust the training parameters to retrain the model. Otherwise, prepare to archive the model uniformly.

[0084] After model conversion, especially for models requiring quantization, the model's confidence threshold will change. A confidence threshold that differs significantly from the original model can affect the effectiveness of model deployment. Therefore, when performing accuracy verification, it's important to consider not only average accuracy metrics such as false positive rate, recall rate, and precision, but also the distribution of confidence thresholds.

[0085] Draw an ROC curve with the false positive rate (FPR, the proportion of samples that are actually negative that are incorrectly predicted as positive by the model) as the horizontal axis and the recall rate (TPR, the proportion of samples that are actually positive that are correctly predicted as positive by the model) as the vertical axis, and sample the ROC curve to obtain the distribution of confidence thresholds under a specific false positive rate.

[0086] Assume that the points that constitute the ROC curve (FPR i , TPR i ) corresponds to the confidence threshold T i , these points are the results of calculations under different confidence thresholds, and the value of a specific FPR is expressed as FPR target , get the corresponding TPR target and T target , sampling of point sets includes the following two cases:

[0087] Case 1, traversal (FPR i , TPR i ) point set. If a point k is found such that FPR k =FPR target , then TPR target =TPR k , T target =T k .

[0088] Case 2, first, ensure (FPR i , TPR i ) First, the point set is based on FPR i Sort by ascending order.

[0089] Find two adjacent points Pt lower = (FPR lower , TPR lower ), threshold T lower and Pt upper = (FPR upper , TPR upper ), threshold T upper . If FPR lower <FPR target <FPR upper , use linear interpolation to calculate TPR target and T target . Linear interpolation formula is expressed as:

[0090] TPR target = TPR lower + (TPR upper - TPR lower )*(FPR target - FPR lower ) / (FPR upper - FPR lower );

[0091] T target = T lower + (T upper - T lower )*(FPR target - FPR lower ) / (FPR upper - FPR lower ).

[0092] The point set on the ROC curve is sampled.

[0093] At the same time, draw the PR curve with precision (Precision, P) as the abscissa and recall (Recall, R) as the abscissa, sample the point set on the PR curve, and obtain the confidence threshold distribution under a certain precision.

[0094] Assume that the points (R i , P i ) constituting the PR curve correspond to the threshold T i , these points are calculated under different confidence thresholds, and the value of a certain P represents P target , obtain the corresponding R target and T target .

[0095] Case one, traverse the (R i , P i ) point set. If a point k is found such that P k == P target , Rtarget = R k , T target = T k .

[0096] Case two, first, make sure the point set on (R i , P i ) is arranged in ascending order of P i .

[0097] Find two adjacent points Pt lower = (R lower , P lower ), threshold T lower and Pt upper = (R upper , P upper ), threshold T upper . If P upper < P target < P upper , use linear interpolation to calculate R target and T target . Linear interpolation formula is expressed as:

[0098] R target = R lower + (R upper - R lower ) * (P target - P lower ) / (P upper - P lower );

[0099] T target = T lower + (T upper - T lower ) * (P target - P lower ) / (P upper - P lower ).

[0100] Thus, the point set on the PR curve is sampled.

[0101] Figure 7 is a storage process schematic diagram of an optional target model according to an embodiment of the present application, as shown in Figure 7As shown, after completing the precision verification process, if it is found that the precision of the conversion model is not lost, the target model is included in the version management process and stored. First, the algorithm type needs to be defined, such as pedestrian detection, flame detection, vehicle detection, etc. If it is not defined in advance, the algorithm type needs to be added. Next, it is determined whether the target model and the existing historical model are converted from the same original algorithm model. If the target model and the historical model are converted from the same original algorithm model, the target model and the historical model version number remain the same, and only the deployment device type is added; otherwise, a new version number needs to be added. In this way, under each version, the conversion model version of the original model for different deployment devices is saved, which facilitates unified management. Moreover, the test set, test precision information and model training log can be recorded and stored, so that the test precision information, test set and training process information of the conversion model can be traced back, the properties of the model can be found, and the confidence threshold can be recommended for model deployment according to the actual use.

[0102] In the embodiment of the application, from data management, model training, model conversion, precision verification to model storage management, end-to-end model development is performed, manual intervention and script execution in the middle process are reduced, the efficiency of algorithm development and deployment is improved, and the possibility of manual operation error is reduced. At the same time, through the standardized precision verification process system, the whole process operation of the test set preparation, model reasoning and index calculation is standardized, and the performance evaluation results under different hardware platforms are ensured to be comparable and traceable. The storage of the associated information of the trained and converted model records the model training, conversion and precision verification results and other information, forms a traceable version, and facilitates the finding of related information of the model.

[0103] The following will be described in detail in combination with another embodiment.

[0104] Embodiment Two

[0105] The target model generation device provided in the embodiment includes a plurality of implementation units, each implementation unit corresponding to each implementation step in the above-mentioned embodiment one, and the specific implementation manner and beneficial effects thereof can refer to the foregoing method embodiment, which will not be described herein again.

[0106] Figure 8 is a schematic diagram of an optional target model generation device according to an embodiment of the application, as Figure 8 shown, the target model generation device can include a construction unit 81, a training unit 82, a conversion unit 83, a test unit 84 and a storage unit 85, wherein,

[0107] The construction unit 81 is configured to parse a generation request of a target model to obtain a project requirement, construct a sample data set for the target model to be generated based on the project requirement, and divide the sample data set to obtain a training set, a test set, and a quantization set, wherein the quantization set is used to adjust quantization parameters of the model in a model quantization process;

[0108] The training unit 82 is configured to construct an initial model based on the project requirement, and iteratively train the initial model based on the training set to obtain a trained initial target model, perform precision testing on the initial target model based on the test set to obtain a precision testing result, and generate a first precision testing result based on the precision testing result of the initial target model, wherein the precision testing result at least includes an average precision indicator, a recall rate, and confidence distribution information;

[0109] The conversion unit 83 is configured to determine a target device to which the target model is to be deployed, and convert the initial target model based on the target device and the quantization set to obtain a converted target model.

[0110] The test unit 84 is configured to perform precision testing on the converted target model based on the test set to obtain a second precision testing result, and compare the first precision testing result with the second precision testing result to obtain a comparison result.

[0111] The storage unit 85 is configured to, in a case where the comparison result indicates that there is no precision loss of the converted target model relative to the initial target model, store the converted target model as a final target model, and store associated information corresponding to the target model, wherein the stored target model is used for deployment of an application, a corresponding converted model is deployed for different to-be-deployed devices, and a confidence threshold of the model deployment model is set according to the associated information.

[0112] The generation device of the target model analyzes the generation request of the target model through the construction unit 81 to obtain the project requirement, constructs a sample data set for the target model to be generated based on the project requirement, and divides the sample data set to obtain a training set, a test set, and a quantization set, wherein the quantization set is used to adjust the quantization parameter of the model in the model quantization process; the training unit 82 constructs an initial model based on the project requirement, and iteratively trains the initial model by using the training set to obtain a trained initial target model, performs precision testing on the initial target model by using the test set to obtain a precision testing result, generates a first precision testing result based on the precision testing result of the initial target model, wherein the precision testing result at least includes: average precision index, recall rate, and confidence distribution information; the conversion unit 83 determines the target device to which the target model is to be deployed, and converts the initial target model based on the target device and the quantization set to obtain a converted target model; the test unit 84 performs precision testing on the converted target model based on the test set to obtain a second precision testing result, and compares the first precision testing result with the second precision testing result to obtain a comparison result; and the storage unit 85 stores the converted target model as the final target model in the case where the comparison result indicates that there is no precision loss of the converted target model relative to the initial target model, and stores the associated information corresponding to the target model, wherein the stored target model is used for deployment application, a corresponding converted model is deployed for different to-be-deployed devices, and the confidence threshold of the deployed model is set according to the associated information.

[0113] In the embodiment, the data set for model training is constructed based on the project requirement, the model is iteratively trained according to the project requirement, the model is converted based on the target device to be deployed, and the converted precision testing and model storage management are provided, thereby providing a full-process generation scheme from the construction of training data, model training, model conversion to model precision testing and model storage management, realizing end-to-end model generation, reducing the risk of unknown precision loss after model conversion through fine precision verification, enhancing the reliability and stability of the model in actual deployment, and thereby solving the technical problem in the related art that the model construction is often focused on some local links of model training or deployment, the fragmentation of the model construction has low automation degree, and the model development efficiency is low, and unknown precision loss is easily generated in model deployment.

[0114] Further, the conversion unit includes: a first acquisition module configured to acquire the device type of the target device, and determine the specific model format after conversion based on the device type; a second acquisition module configured to acquire the hardware specification information of the target device, and determine the quantization precision of the target device based on the hardware specification information; and a first conversion module configured to convert the initial target model in a general format into a converted target model conforming to the specific model format by using the quantization set and based on the model format and the quantization precision.

[0115] Further, the first conversion module comprises: a first collection submodule, configured to input the quantization set as input data into the initial target model, and collect the dynamic range and the quantization parameter of each network layer in the initial target model based on an output result of the initial target model and a quantization precision, wherein the dynamic range at least comprises a weight range and an activation value range, and the quantization parameter at least comprises a scaling factor and a zero point; a first conversion submodule, configured to convert the weight of each network layer of the initial target model from a floating point number to an integer representation based on the scaling factor, the zero point and the weight range; a first quantization submodule, configured to quantize the activation value of each network layer based on the activation value range in a forward propagation process of the initial model, to obtain a quantization target model; and a first conversion submodule, configured to convert the quantization target model from a general format into a specific model format, to obtain a conversion target model.

[0116] Further, the test unit comprises: a first acquisition module, configured to input the test set as input data into the conversion target model, and acquire an average precision index of the conversion target model based on an output result of the conversion target model; a first drawing module, configured to draw a first reference curve of the conversion target model, and sample points on the first reference curve to obtain a recall rate corresponding to a fixed false detection rate, wherein the first reference curve is a curve composed of a false detection rate and a recall rate; and a second drawing module, configured to draw a second reference curve of the conversion target model, and sample points on the second reference curve to obtain a recall rate corresponding to a fixed precision, wherein the second reference curve is a curve composed of a precision and a recall rate.

[0117] Further, the test unit further comprises: a first comparison module, configured to compare the average precision index of the first precision test result with the average precision index of the second precision test result, to obtain a first comparison result; a second comparison module, configured to compare the recall rate corresponding to the fixed false detection rate in the first precision test result with the recall rate corresponding to the fixed false detection rate in the second precision test result, to obtain a second comparison result; a third comparison module, configured to compare the recall rate corresponding to the fixed precision in the first precision test result with the recall rate corresponding to the fixed precision in the second precision test result, to obtain a third comparison result; and a third acquisition module, configured to obtain a comparison result based on the first comparison result, the second comparison result and the third comparison result.

[0118] Further, the storage unit comprises: a first determination module configured to determine that the conversion target model passes the average precision index test if a difference between the average precision index of the first precision test result and the average precision index of the second precision test result indicated by the first comparison result is within a precision index difference range; a second determination module configured to determine that the conversion target model passes the recall rate test if a difference between the recall rate corresponding to the fixed false detection rate in the first precision test result and the recall rate corresponding to the fixed false detection rate in the second precision test result indicated by the second comparison result is within a first preset range, and if a difference between the recall rate corresponding to the fixed precision rate in the first precision test result and the recall rate corresponding to the fixed precision rate in the second precision test result indicated by the third comparison result is within a second preset range; a third determination module configured to determine that the conversion target model does not have precision loss relative to the initial target model if the conversion target model passes the average precision index test and the recall rate test, obtain a target model, and store the target model; and a second conversion module configured to re-convert the initial target model if the conversion target model does not pass the average precision index test and / or the recall rate test.

[0119] Further, the storage unit further comprises: a first definition module configured to define an algorithm type of the target model; a first determination module configured to determine whether the target model and a historical model are converted from the same initial target model, obtain a determination result, and configure a version number for the target model based on the determination result; a second acquisition module configured to acquire precision test information, a test set and a training log of the target model; and a first storage module configured to obtain associated information corresponding to the target model based on the algorithm type, the version number, the precision test information, the test set and the training log of the target model, and store the associated information.

[0120] According to another aspect of the embodiments of the present application, a computer readable storage medium is also provided, which comprises a stored computer program, wherein the computer program, when executed, controls a device where the computer readable storage medium is located to perform the generation method of any one of the target models.

[0121] According to another aspect of the embodiments of the present application, an electronic device is also provided, which comprises one or more processors and a memory, the memory being configured to store one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the generation method of any one of the target models.

[0122] In the present application, by the following steps: analyzing the generation request of the target model, obtaining the project requirement, and constructing a sample data set for the target model to be generated based on the project requirement, and dividing the sample data set to obtain a training set, a test set and a quantization set, wherein the quantization set is used to adjust the quantization parameters of the model in the model quantization process, and constructing an initial model based on the project requirement, and iteratively training the initial model using the training set to obtain a trained initial target model, using the test set to test the accuracy of the initial target model to obtain an accuracy test result, and generating a first accuracy test result based on the accuracy test result of the initial target model, wherein the accuracy test result at least includes: average accuracy index, recall rate, confidence distribution information, then determining the target device to which the target model is to be deployed, and transforming the initial target model based on the target device and the quantization set to obtain a transformed target model, and testing the accuracy of the transformed target model based on the test set to obtain a second accuracy test result, and comparing the first accuracy test result with the second accuracy test result to obtain a comparison result, and finally in the case that the comparison result indicates that there is no accuracy loss of the transformed target model relative to the initial target model, storing the transformed target model as the final target model, and storing the associated information corresponding to the target model, wherein the stored target model is used for deploying an application, and deploying a corresponding transformed model for different to-be-deployed devices, and setting a confidence threshold of the deployed model according to the associated information.

[0123] It should be noted that the above-mentioned construction unit 81, training unit 82, transformation unit 83, test unit 84 and storage unit 85 correspond to steps S201 to S205 in Embodiment One, and the above-mentioned units have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment One. It should be noted that the above-mentioned modules or units can be hardware components or software components stored in a memory (for example, memory 104) and processed by one or more processors (for example, processors 102a, 102b, …, 102n), and the above-mentioned modules or units can also be run in the computer terminal 10 provided in Embodiment One as part of the device.

[0124] The present application will be described below in conjunction with another alternative embodiment.

[0125] Embodiment Three

[0126] The embodiments of the present application can also provide an electronic device, Figure 9 is a hardware structure block diagram of an electronic device (or mobile device) according to an alternative execution target model generation method according to an embodiment of the present application, as Figure 9 shown, the electronic device can include one or more Figure 9The processor 902, the memory 904, the storage controller, and the peripheral interface are shown in FIG. 1. The peripheral interface is connected with a radio frequency module, an audio module, and a display.

[0127] The memory can be used to store software programs and modules, such as program instructions / modules corresponding to the method and device in the embodiments of the present application. The processor executes various functions and data processing by running the software programs and modules stored in the memory, that is, implements the above method. The memory can include a high-speed random access memory, and can further include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory can further include a memory remotely arranged with respect to the processor, which can be connected to the terminal through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0128] The processor can call information and applications stored in the memory through the transmission device to perform the following steps: parsing a generation request of a target model to obtain project requirements, and constructing a sample data set for the target model to be generated based on the project requirements, and dividing the sample data set to obtain a training set, a test set, and a quantization set, wherein the quantization set is used to adjust the quantization parameters of the model in the model quantization process; constructing an initial model based on the project requirements, and iteratively training the initial model using the training set to obtain a trained initial target model, testing the accuracy of the initial target model using the test set to obtain an accuracy test result, and generating a first accuracy test result based on the accuracy test result of the initial target model, wherein the accuracy test result at least includes: average accuracy index, recall rate, and confidence distribution information; determining a target device to which the target model is to be deployed, and transforming the initial target model based on the target device and the quantization set to obtain a transformed target model; testing the accuracy of the transformed target model based on the test set to obtain a second accuracy test result, and comparing the first accuracy test result with the second accuracy test result to obtain a comparison result; in the case that the comparison result indicates that there is no accuracy loss of the transformed target model relative to the initial target model, storing the transformed target model as a final target model, and storing associated information corresponding to the target model, wherein the stored target model is used for deployment application, and a corresponding transformed model is deployed for different to-be-deployed devices, and the confidence threshold of the deployed model is set according to the associated information.

[0129] The embodiment of the present application provides a scheme for automatic generation of a whole process of a target model. The scheme comprises the following steps: constructing a data set for model training based on project requirements, iteratively training the model according to the project requirements, transforming the model based on a target device to be deployed, and performing precision testing and model storage management on the transformed model, thereby providing a whole-process generation scheme from construction of training data, model training, model transformation to model precision testing and model storage management, realizing end-to-end model generation, reducing the risk of unknown precision loss after model transformation through fine precision verification, and enhancing the reliability and stability of the model in actual deployment, thereby solving the technical problem in the prior art that model training or deployment is often focused on some local links, the split model construction has low automation, and the model development efficiency is low and unknown precision loss is prone to occur during model deployment.

[0130] Those skilled in the art can understand that, Figure 9 The structure shown is only schematic, and the electronic device can also be a smart phone, a tablet computer, a palm computer, a mobile Internet device (MID), a PAD, or the like. Figure 9 The structure of the electronic device is not limited. For example, the electronic device can further include more or fewer components (such as a network interface, a display device, or the like) than those shown in the figure, or have a different configuration from that shown in the figure. Figure 9 The structure of the electronic device is not limited. For example, the electronic device can further include more or fewer components (such as a network interface, a display device, or the like) than those shown in the figure, or have a different configuration from that shown in the figure. Figure 9 The structure of the electronic device is not limited. For example, the electronic device can further include more or fewer components (such as a network interface, a display device, or the like) than those shown in the figure, or have a different configuration from that shown in the figure.

[0131] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by programs instructing the hardware of the terminal device, and the programs can be stored in a computer readable storage medium, which can include a flash disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and the like.

[0132] The present application will be described below in combination with another alternative embodiment.

[0133] Embodiment Four

[0134] The embodiment of the present application further provides a computer readable storage medium. Optionally, in the embodiment of the present application, the above computer readable storage medium can be used to save the program code executed by the target model generation method provided in the above embodiment one.

[0135] Optionally, in the embodiment of the present application, the above storage medium can be located in any one of the computer terminals in the computer terminal group in the computer network, or in any one of the mobile terminals in the mobile terminal group.

[0136] The embodiment of the present application also provides a computer program product, when executed on a data processing device, is adapted to execute the steps of the generation method of the target model: parsing a generation request of the target model, obtaining a project requirement, and constructing a sample data set for the target model to be generated based on the project requirement, and dividing the sample data set to obtain a training set, a test set and a quantization set, wherein the quantization set is used to adjust a quantization parameter of the model in a model quantization process; constructing an initial model based on the project requirement, and iteratively training the initial model using the training set to obtain a trained initial target model, performing precision testing on the initial target model using the test set to obtain a precision testing result, and generating a first precision testing result based on the precision testing result of the initial target model, wherein the precision testing result at least includes: an average precision index, a recall rate and confidence distribution information; determining a target device to which the target model is to be deployed, and transforming the initial target model based on the target device and the quantization set to obtain a transformed target model; performing precision testing on the transformed target model based on the test set to obtain a second precision testing result, and comparing the first precision testing result with the second precision testing result to obtain a comparison result; in the case that the comparison result indicates that there is no precision loss of the transformed target model relative to the initial target model, storing the transformed target model as a final target model, and storing associated information corresponding to the target model, wherein the stored target model is used for deploying an application, a corresponding transformed model is deployed for different to-be-deployed devices, and a confidence threshold of the deployed model is set according to the associated information.

[0137] The above embodiment numbers of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0138] In the above embodiments of the present application, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0139] In several embodiments provided in the present application, it should be understood that the disclosed technical contents can be implemented by other ways. Among them, the above-mentioned device embodiments are only schematic, for example, the division of the units can be a logical function division, and actual implementation can have another division way, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be through some interface, indirect coupling or communication connection between units or modules, which can be electrical or other forms.

[0140] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0141] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0142] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical scheme of the present application or the part of the present application which contributes to the prior art or the whole or part of the technical scheme can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes various media that can store program codes, such as a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0143] The above is only the preferred embodiment of the present application, and it should be pointed out that for ordinary skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should be considered as the protection scope of the present application.

Claims

1. A method for generating a target model, characterized in that: include: Parsing the target model generation request to obtain project requirements, and constructing a sample data set for the target model to be generated based on the project requirements, and dividing the sample data set into a training set, a test set, and a quantization set, wherein the quantization set is used to adjust the model's quantization parameters during the model quantization process; Constructing an initial model based on the project requirements, and iteratively training the initial model using the training set to obtain a trained initial target model, performing an accuracy test on the initial target model using the test set to obtain an accuracy test result, and generating a first accuracy test result based on the accuracy test result of the initial target model, wherein the accuracy test result at least includes: an average accuracy index, a recall rate, and confidence distribution information; Determining a target device on which the target model is to be deployed, and transforming the initial target model based on the target device and the quantization set to obtain a transformed target model; Performing an accuracy test on the conversion target model based on the test set to obtain a second accuracy test result, and comparing the first accuracy test result with the second accuracy test result to obtain a comparison result; When the comparison result indicates that there is no loss of accuracy in the conversion target model relative to the initial target model, the conversion target model is stored as the final target model, and the associated information corresponding to the target model is stored, wherein the stored target model is used to deploy the application, and the corresponding conversion model is deployed for different devices to be deployed, and the confidence threshold of the deployment model is set according to the associated information.

2. The method according to claim 1, characterized in that The step of converting the initial target model based on the target device and the quantization set includes: Acquiring a device type of the target device, and determining a specific model format after conversion based on the device type; Acquiring hardware specification information of the target device, and determining a quantization accuracy of the target device based on the hardware specification information; The quantization set is used, and based on the model format and the quantization accuracy, the initial target model in a universal format is converted into the converted target model that complies with the specific model format.

3. The method according to claim 2, characterized in that The step of converting the initial target model in a general format into the converted target model conforming to a specific model format by using the quantization set and based on the model format and the quantization accuracy includes: Using the quantization set as input data to input into the initial target model, and collecting the dynamic range and quantization parameters of each network layer in the initial target model based on the output result of the initial target model and the quantization accuracy, wherein the dynamic range includes at least: a weight range and an activation value range, and the quantization parameters include at least: a scaling factor and a zero point; Based on the scaling factor, the zero point, and the weight range, converting the weights of each network layer of the initial target model from floating point numbers to integer representations; During the forward propagation of the initial model, the activation values ​​of each of the network layers are quantized based on the activation value range to obtain a quantized target model; The quantitative target model is converted from a general format into the specific model format to obtain the converted target model.

4. The method according to claim 1, wherein The step of performing accuracy testing on the conversion target model based on the test set includes: The test set is input into the conversion target model as input data, and the average precision index of the conversion target model is collected based on the output result of the conversion target model. Draw a first reference curve of the conversion target model, and sample points on the first reference curve to obtain a recall rate corresponding to a fixed false positive rate, wherein the first reference curve is a curve composed of the false positive rate and the recall rate; A second reference curve of the conversion target model is drawn, and points on the second reference curve are sampled to obtain a recall rate corresponding to a fixed precision rate, wherein the second reference curve is a curve composed of precision rate and recall rate.

5. The method according to claim 1, wherein The step of comparing the first accuracy test result with the second accuracy test result to obtain a comparison result includes: Comparing the average precision index of the first precision test result with the average precision index of the second precision test result to obtain a first comparison result; Comparing the recall rate corresponding to the fixed false positive rate in the first precision test result with the recall rate corresponding to the fixed false positive rate in the second precision test result to obtain a second comparison result; Comparing the recall rate corresponding to the fixed precision rate in the first precision test result with the recall rate corresponding to the fixed precision rate in the second precision test result to obtain a third comparison result; The comparison result is obtained based on the first comparison result, the second comparison result, and the third comparison result.

6. The method according to claim 5, characterized in that When the comparison result indicates that the converted target model has no accuracy loss relative to the initial target model, the step of storing the converted target model as the final target model includes: If the first comparison result indicates that a difference between an average precision index of the first precision test result and an average precision index of the second precision test result is within an accuracy index difference range, determining that the conversion target model passes the average precision index test; If the second comparison result indicates that the difference between the recall rate corresponding to the fixed false positive rate in the first precision test result and the recall rate corresponding to the fixed false positive rate in the second precision test result is within a first preset range, and the third comparison result indicates that the difference between the confidence threshold distribution corresponding to the fixed precision rate in the first precision test result and the confidence threshold distribution corresponding to the fixed precision rate in the second precision test result is within a second preset range, it is determined that the conversion target model passes the recall rate test; If the converted target model passes the average precision index test and the recall rate test, determining that the converted target model has no accuracy loss relative to the initial target model, obtaining the target model, and storing the target model; or If the converted target model fails the average precision index test and / or the recall rate test, the initial target model is converted again.

7. The method according to claim 5, characterized in that The step of storing the associated information corresponding to the target model includes: defining the algorithm type of the target model; Determining whether the target model and the historical model are converted from the same initial target model, obtaining a determination result, and configuring a version number for the target model based on the determination result; Collecting accuracy test information, test sets, and training logs of the target model; Based on the algorithm type of the target model, the version number, the accuracy test information, the test set and the training log, the associated information corresponding to the target model is obtained, and the associated information is stored.

8. A target model generation device, characterized in that: include: A construction unit is configured to parse a target model generation request to obtain project requirements, construct a sample data set for the target model to be generated based on the project requirements, and divide the sample data set into a training set, a test set, and a quantization set, wherein the quantization set is used to adjust the quantization parameters of the model during the model quantization process; A training unit is configured to construct an initial model based on the project requirements, and iteratively train the initial model using the training set to obtain a trained initial target model, perform an accuracy test on the initial target model using the test set to obtain an accuracy test result, and generate a first accuracy test result based on the accuracy test result of the initial target model, wherein the accuracy test result includes at least: an average accuracy index, a recall rate, and confidence distribution information; a conversion unit, configured to determine a target device on which the target model is to be deployed, and convert the initial target model based on the target device and the quantization set to obtain a converted target model; a testing unit, configured to perform an accuracy test on the conversion target model based on the test set to obtain a second accuracy test result, and compare the first accuracy test result with the second accuracy test result to obtain a comparison result; A storage unit is used to store the conversion target model as the final target model when the comparison result indicates that there is no loss of accuracy in the conversion target model relative to the initial target model, and to store the associated information corresponding to the target model, wherein the stored target model is used to deploy applications, and corresponding conversion models are deployed for different devices to be deployed, and a confidence threshold of the deployment model is set according to the associated information.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the method for generating a target model according to any one of claims 1 to 7.

10. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method for generating a target model as described in any one of claims 1 to 7.

Images