Quantification method and apparatus for image segmentation model, and device, medium and program product

Abstract

The present application relates to the technical field of computers. Provided are a quantification method and apparatus for an image segmentation model, and a device and a computer storage medium. The method comprises: acquiring a computing power evaluation result of a device to be deployed, and a model to be quantified, wherein the computing power evaluation result is used for indicating the operation performance of the device to be deployed, and said model is used for implementing image segmentation; on the basis of the computing power evaluation result, selecting at least one target operator from operators included in said model, wherein a data processing duration of the target operator on the device to be deployed reaches a set threshold value; performing quantification processing on the at least one target operator on the basis of the difference between the obtained individual data processing duration of the at least one target operator and a corresponding expected processing duration, so as to acquire a corresponding quantification operator, wherein a data processing duration of the quantification operator on the device to be deployed is less than the expected processing duration; and converting said model into a corresponding target model on the basis of at least one quantification operator.

Description

Quantization method, device, equipment, medium and program product of image segmentation model

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on July 25, 2023, with application number 2023109160698 and application name “Quantization method, device and equipment of image segmentation model and computer storage medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to the quantization of image segmentation models. Background Art

[0003] With the continuous development of Internet technology, various image segmentation models have been able to achieve more refined and accurate image segmentation results. However, achieving such results often relies on high hardware costs. If these models are directly applied to low-cost hardware devices, it will be difficult to achieve the expected results.

[0004] For example, in some image recognition processes, the image must first be segmented and then feature extracted from the segmented image to achieve image recognition. This process consumes a lot of computing power. To properly implement this process, the hardware that supports it must have high computing power. As a result, high-precision image recognition is difficult to achieve on low-cost hardware devices, reducing the efficiency of its widespread application.

[0005] In particular, iris recognition technology, used in virtual reality (VR) devices, often requires high hardware costs and complex software design, as the segmentation algorithms used in the image segmentation step of iris recognition are often deployed in embedded systems. Consequently, when deployed on low-cost hardware, the iris segmentation step takes a long time to complete, making it difficult to achieve the desired real-time performance. This makes it difficult to deploy iris recognition technology on low-cost hardware devices, hindering the demand for its large-scale application.

[0006] Therefore, how to improve the deployment effect of image segmentation models on low-cost hardware devices is an urgent problem to be solved.

[0007] Summary of the Invention

[0008] The present application provides a method, apparatus, and device for image segmentation, as well as a computer storage medium and program product, for improving the deployment effect of image segmentation models on low-cost hardware devices.

[0009] In a first aspect, the present application provides a quantization method for an image segmentation model, the method comprising:

[0010] Obtaining computing power evaluation results of the devices to be deployed and the models to be quantified; the computing power evaluation results are used to indicate the computing performance of the devices to be deployed, and the models to be quantified are used to implement image segmentation;

[0011] Based on the computing power evaluation results, select at least one target operator from the operators included in the model to be quantified, and ensure that the data processing time of the target operator on the device to be deployed reaches the set threshold.

[0012] Based on the obtained difference between the data processing time of each of the at least one target operator and the corresponding expected processing time, quantize the at least one target operator to obtain a corresponding quantized operator, where the data processing time of the quantized operator on the device to be deployed is less than the expected processing time;

[0013] Based on at least one quantization operator, the model to be quantized is converted into a corresponding target model, where the target model is used to be deployed to the device to be deployed.

[0014] In a second aspect, the present application provides a quantization device for an image segmentation model, the device comprising:

[0015] An acquisition module is used to obtain the computing power evaluation results of the device to be deployed and the model to be quantized; wherein the computing power evaluation results are used to indicate the computing performance of the device to be deployed, and the model to be quantized is used to implement image segmentation;

[0016] A screening module is used to select at least one target operator from the operators included in the model to be quantified based on the computing power evaluation results, and the data processing time of the target operator on the device to be deployed reaches a set threshold;

[0017] A quantization module is configured to quantize at least one target operator based on the difference between the obtained data processing time of each target operator and the corresponding expected processing time, thereby obtaining a corresponding quantization operator, wherein the data processing time of the quantization operator on the device to be deployed is less than the expected processing time; and based on the at least one quantization operator, convert the model to be quantized into a corresponding target model, wherein the target model is used to be deployed on the device to be deployed.

[0018] In a third aspect, the present application provides an electronic device comprising a processor and a memory, wherein the memory stores program code, and when the program code is executed by the processor, the processor implements the steps of the quantization method of any one of the above-mentioned image segmentation models.

[0019] In a fourth aspect, the present application also provides a computer-readable storage medium for storing a computer program. When the computer program runs on an electronic device, the computer program is used to enable the electronic device to execute the steps of the quantization method of any one of the image segmentation models.

[0020] In a fifth aspect, the present application also provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the quantization method of any of the above-mentioned image segmentation models.

[0021] The beneficial effects of this application are as follows:

[0022] The embodiment of the present application provides a quantization method, device, equipment and storage medium for an image segmentation model. The method includes: obtaining the computing power evaluation result of the device to be deployed, determining the running status of the model to be quantized on the device to be deployed based on the computing power evaluation result, and thus performing targeted quantization processing on each operator in the model to be quantized, thereby obtaining a quantization operator whose data processing time meets the requirements. In this way, the operator is used as the quantization granularity, which reduces the number of operations when quantizing the model and improves the corresponding quantization efficiency. In addition, a corresponding target model is constructed based on these quantization operators to ensure that when the target model runs on the device to be deployed, the time for processing the image segmentation task can be effectively reduced to the total expected processing time, thereby improving its deployment effect on low-cost hardware and effectively improving the popularity of the image segmentation model on low-cost devices.

[0023] During the process of quantizing the model, the accuracy of the model can be adjusted based on the preset accuracy requirements, and the accuracy of the model can be adjusted to a level that matches the computing power of the device to be deployed, further improving the deployment effect of the target model on the device to be deployed.

[0024] Other features and advantages of the present application will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present application. The purposes and other advantages of the present application can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] FIG1A is a schematic diagram of image segmentation provided in an embodiment of the present application;

[0026] FIG1B is a schematic diagram of a possible application scenario provided by an embodiment of the present application;

[0027] FIG2 is a flow chart of a quantization method for an image segmentation model provided in an embodiment of the present application;

[0028] FIG3 is a flowchart of a method for obtaining computing power evaluation results provided in an embodiment of the present application;

[0029] FIG4 is a schematic diagram of obtaining a computing power evaluation result provided in an embodiment of the present application;

[0030] FIG5 is a schematic diagram of a target operator screening method provided in an embodiment of the present application;

[0031] FIG6 is a schematic diagram of another target operator screening method provided in an embodiment of the present application;

[0032] FIG7 is a flow chart of a target operator screening method provided in an embodiment of the present application;

[0033] FIG8 is a flow chart of a method for adjusting model accuracy provided in an embodiment of the present application;

[0034] FIG9 is a schematic diagram of a method for adjusting model accuracy provided in an embodiment of the present application;

[0035] FIG10A is a flowchart of a method for training a model to be calibrated provided in an embodiment of the present application;

[0036] FIG10B is a schematic diagram of a training model to be calibrated provided in an embodiment of the present application;

[0037] FIG11 is a flowchart of an image segmentation method provided in an embodiment of the present application;

[0038] FIG12 is a schematic diagram of an image segmentation method provided in an embodiment of the present application;

[0039] FIG13A is a flowchart of an image segmentation method provided in an embodiment of the present application;

[0040] FIG13B is a schematic diagram of an image segmentation method provided in an embodiment of the present application;

[0041] FIG14 is a flow chart of a quantization method for an image segmentation model provided in an embodiment of the present application;

[0042] FIG15 is a schematic diagram of a quantization method for an image segmentation model provided in an embodiment of the present application;

[0043] FIG16 is a schematic diagram of the structure of a quantization device for an image segmentation model provided in an embodiment of the present application;

[0044] FIG17 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present application;

[0045] FIG18 is a schematic diagram of a hardware structure of another electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other in any way. In addition, although a logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in an order different from that here.

[0047] It is understood that in the following specific implementations of this application, sample images and other related data are involved. When the various embodiments of this application are applied to specific products or technologies, relevant licenses or consents need to be obtained, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions. For example, when it is necessary to obtain relevant data, relevant volunteers can be recruited and relevant agreements on volunteer authorization data can be signed, and then the data of these volunteers can be used for implementation; or, by implementing within the scope of an authorized organization, the following implementation method can be implemented by using the data of internal members of the organization to perform relevant identification to internal members; or, the relevant data used in the specific implementation are all simulated data, such as simulated data generated in a virtual scene.

[0048] To facilitate understanding of the technical solutions provided in the embodiments of the present application, some key terms used in the embodiments of the present application are explained here:

[0049] Image segmentation is a typical computer vision problem that involves taking some raw data (e.g., a planar image) as input and converting it into a mask with highlighted regions of interest. As shown in Figure 1A, when performing image segmentation on an original image a (top of Figure 1A) containing images of different categories, if the category to be highlighted is limited to vehicles, then after processing by the image segmentation model, the resulting image segmentation result, as shown in segmentation result b (bottom of Figure 1A), highlights the outlines of two vehicles in the panoramic image.

[0050] Operators (OPs) are computational units in deep learning algorithms. In neural network models, operators correspond to the computational logic within a layer. For example, a convolution layer can be an operator, and the weight summation process within a fully-connected layer (FC layer) can also be an operator.

[0051] Floating-point operations per second (GFLOPS): Also known as peak speed per second, it refers to the number of floating-point operations performed per second. Floating-point numbers are numbers with decimals, and floating-point operations are arithmetic operations on decimals. They are often used to measure computer speed or estimate performance, especially in scientific computing, which uses a large number of floating-point operations.

[0052] Floating Point Operations (FLOPs): can be used to measure the complexity of the model / algorithm.

[0053] The following is a brief introduction to the design concept of the embodiment of this application:

[0054] With the development of internet technology, more and more network models are achieving higher-precision technical results. For example, image recognition models can more accurately identify a target image from multiple images, and image segmentation models can more accurately segment images. However, the realization of these technical results currently relies on high hardware costs. If these network models are directly applied to low-cost hardware, it will be difficult to achieve the expected results.

[0055] For example, in some image recognition processes, the image must first be segmented and then feature extracted from the segmented image to achieve image recognition. This process consumes a lot of computing power. To properly implement this process, the hardware that supports it must have high computing power. As a result, high-precision image recognition is difficult to achieve on low-cost hardware devices, reducing the efficiency of its widespread application.

[0056] In particular, iris recognition technology, used in virtual reality (VR) devices, often requires high hardware costs and complex software design, as the segmentation algorithms used in the image segmentation step of iris recognition are often deployed in embedded systems. Consequently, when deployed on low-cost hardware, the iris segmentation step takes a long time to complete, making it difficult to achieve the desired real-time performance. This makes it difficult to deploy iris recognition technology on low-cost hardware devices, hindering the demand for its large-scale application.

[0057] In view of this, an embodiment of the present application provides a quantization method for an image segmentation model to improve the deployment effect of the image segmentation model on low-cost hardware devices.

[0058] This method first performs a computing power evaluation on the device to be deployed where the image segmentation model is planned to be deployed, obtains a computing power evaluation result indicating the computing performance of the device to be deployed, and obtains a model to be quantified for implementing image segmentation. Based on the computing power evaluation result for the device to be deployed, at least one target operator whose data processing time on the device to be deployed reaches a set threshold is selected from the operators included in the model to be quantified. Based on the difference between the target operator's data processing time and its corresponding expected processing time, the target operator is quantized to obtain the corresponding quantized operator. Then, based on these quantized operators, the model to be quantized is converted into a corresponding target model. In this way, after quantizing each operator in the model to be quantified, the processing time of the target model obtained when performing image segmentation on the device to be deployed can meet the expected requirements, effectively improving its deployment effect on low-cost hardware devices.

[0059] Furthermore, after quantization processing is performed on each operator, a corresponding intermediate model can be constructed based on these quantized operators and other operators in the model to be quantized; and, based on the computing power evaluation results and the total expected processing time corresponding to the target model, the target accuracy that the target model should correspond to is determined. In this way, based on the target accuracy, the model accuracy corresponding to the intermediate model can be adjusted so that its model accuracy is the same as the target accuracy, thereby obtaining a target model whose processing time for completing the image segmentation task on the device to be deployed meets the total expected processing time.

[0060] At the same time, after completing the accuracy conversion of the intermediate model, the model to be calibrated after the accuracy conversion can be set in an operating environment with the same hardware parameters as the device to be deployed, and based on some sample segmentation images and sample segmentation results, the model parameters of the model to be calibrated can be adjusted, so that the target model after parameter adjustment can take into account the accuracy and efficiency of image segmentation.

[0061] In the embodiments of the present application, the parts of obtaining the model to be quantized, the quantization operator, and adjusting the model parameters involve artificial intelligence (AI), computer vision technology (CV), and machine learning (ML), and are designed based on image segmentation technology and machine learning in artificial intelligence.

[0062] Artificial intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also involves studying the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0063] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, pre-trained models, operating / interaction systems, and mechatronics. Pre-trained models, also known as large models or basic models, can be fine-tuned and widely applied to downstream tasks across various AI disciplines. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0064] Computer vision is the science of making machines "see." Specifically, it refers to using cameras and computers to replace the human eye in identifying, tracking, and measuring objects, and then further processing the images to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, aiming to build artificial intelligence systems that can extract information from images or multidimensional data. Large model technology has brought significant changes to the development of computer vision technology. Pre-trained models in the field of vision, such as the Swin Transformer, ViT, V-MOE, and MAE, can be fine-tuned to quickly and widely apply to specific downstream tasks. Computer vision technologies generally include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / action recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0065] For example, in an embodiment of the present application, the target model can implement image processing of the iris image to be segmented, thereby identifying the area where the iris is located.

[0066] Machine learning is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning. Pretrained models are the latest development in deep learning, integrating these techniques.

[0067] For example, in an embodiment of the present application, the model to be quantized can be obtained through model training, and the accuracy evaluation and parameter adjustment of the model to be calibrated can be achieved through a sample set.

[0068] The following briefly introduces the application scenarios to which the technical solutions of the embodiments of the present application can be applied. It should be noted that the application scenarios described below are only used to illustrate the embodiments of the present application and are not limiting. In the specific implementation process, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0069] The solution provided in the embodiments of the present application can be applied to most scenarios where image segmentation models are deployed, for example, scenarios where image segmentation is performed during face recognition, and scenarios where iris segmentation is performed during iris recognition in VR devices.

[0070] 1B , which is a schematic diagram of a possible application scenario provided in an embodiment of the present application. In this scenario, a terminal device 101 and a server 102 may be included.

[0071] The terminal device 101 can be a mobile phone, tablet computer (PAD), personal computer (PC), vehicle-mounted terminal, etc., or a camera, camcorder, driving recorder, etc., or a wearable device such as VR glasses and smart watches. The server 102 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0072] The server 102 may include one or more processors 1021, a memory 1022, and an I / O interface 1023 for interacting with a terminal. In addition, the server 102 may also be configured with a database 1024, which may be used to store images to be segmented, image segmentation results, sample sets, model parameters of the model to be quantized, and model parameters of the target model. The memory 1022 of the server 102 may also store program instructions for the image model quantization method provided in the embodiment of the present application. When these program instructions are executed by the processor 1021, they may be used to implement the steps of the image model quantization method provided in the embodiment of the present application to obtain a target model and image segmentation results, etc., and the target model may be deployed or the target model effect may be displayed on the terminal device 101 accordingly.

[0073] The terminal device 101 and the server 102 can be directly or indirectly connected to each other through one or more communication networks 103. The communication network 103 can be a wired network or a wireless network, for example, a mobile cellular network or a Wireless Fidelity (WIFI) network, or other possible networks, which are not limited in the present embodiment.

[0074] It should be noted that the quantization method of the image model in the embodiment of the present application can be executed by a computer device, which can be a terminal device 101 or a server 102, that is, the method can be executed by the terminal device 101 or the server 102 alone, or can be executed jointly by the terminal device 101 and the server 102.

[0075] For example, when the method is executed by server 102 alone, server 102 can obtain the corresponding model to be quantified and the computing power evaluation results of the device to be deployed, and then screen the target operator of the model to be quantified and quantify the target operator based on the model to be quantified and the computing power evaluation results, and then obtain the target model.

[0076] For another example, when the terminal device 101 executes this method alone, there may be two situations: the terminal device is a device to be deployed and the terminal device is not a device to be deployed. In this way, when the terminal device executes this method, it can be divided into two types: directly obtaining its own computing power evaluation results and obtaining its computing power evaluation results from other terminal devices. Other execution methods are similar to server execution.

[0077] For another example, the method can also be performed jointly by the terminal device 101 and the server 102. The terminal device can provide the server with a sample set or an image to be segmented or a computing power evaluation result, and the server can quantize the target operator in the model to be quantized based on the obtained computing power evaluation result and the model to be quantized, thereby obtaining the corresponding target model.

[0078] It should be noted that what is shown in FIG1B is only an example. In fact, the number and communication mode of terminal devices and servers are not limited and are not specifically limited in the embodiments of this application.

[0079] The following describes the quantization method of the image segmentation model provided by the exemplary embodiment of the present application in combination with the application scenarios described above and with reference to the accompanying drawings. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principles of the present application, and the implementation of the present application is not limited in this respect.

[0080] See Figure 2, which is a flowchart of a quantization method for an image segmentation model provided in an embodiment of the present application. This method can be executed by the aforementioned computer device. In this embodiment, for ease of explanation, the following description will be based on the example of the computer device being a server. The specific operating steps of this method are as follows:

[0081] S201: Obtain a computing power evaluation result of a device to be deployed and a model to be quantified; wherein the computing power evaluation result is used to indicate the computing performance of the device to be deployed, and the model to be quantified is used to implement image segmentation.

[0082] Specifically, when executing the image segmentation model quantization method, the server first needs to obtain the image segmentation model to be quantized (hereinafter referred to as the quantized model) and the computing power evaluation results of the device where the quantized model will be deployed (the deployed device). There are several different ways to obtain the quantized model.

[0083] Exemplarily, before executing the quantization method, the server can directly train an image segmentation model capable of image segmentation based on a convolutional neural network as a subsequent model to be quantized; wherein, during the training process of the image segmentation model, the model convergence condition can be set according to the needs of the user, and this application does not limit this. After the model training process, the image segmentation model obtained can be directly used for image segmentation, and the accuracy and efficiency of its image segmentation can also achieve very good results. The premise for achieving this effect is that the image segmentation model runs on a relatively high-cost hardware device. If it is expected that the image segmentation model can effectively run on low-cost hardware, the server needs to execute the quantization method of the image segmentation model provided in the embodiment of the present application on the image segmentation model.

[0084] For another example, the server can directly obtain some image segmentation models that have been trained. If it is determined that these models that have been trained and can directly perform image segmentation also take a long time to run on low-cost hardware, the server can execute the image segmentation model quantization method provided in the embodiment of the present application.

[0085] After the image segmentation models are obtained in the above-mentioned different ways, a quantization method needs to be performed on them. These image segmentation models are referred to as models to be quantized.

[0086] Because different devices have different hardware parameters, if you want the model to perform well on all the devices you want to deploy, the server needs to perform different quantization processes on the model based on the hardware parameters of each device. This ensures that the model performs well on all the devices. The following describes how to perform quantization on a specific device, using a specific device as an example.

[0087] After obtaining the model to be quantified, the server needs to evaluate the computing power of the device to be deployed.

[0088] Optionally, when obtaining the computing power evaluation result of a device to be deployed, the processing method shown in FIG3 may be executed thereon.

[0089] Referring to FIG3 , which is a flow chart of a method for obtaining computing power evaluation results provided in an embodiment of the present application, as shown in FIG3 , the method specifically performs the following steps:

[0090] S301: Performing a data processing test on the device to be deployed based on multiple random data, and obtaining a test processing time required for the device to be deployed to perform the data processing test.

[0091] S302: Based on the test processing time and the data volume corresponding to the multiple random data, obtain the computing power evaluation result corresponding to the device to be deployed.

[0092] Specifically, when the server needs to perform a computing power test on the device to be deployed, the focus of the test is the computing power of the processor used for data calculation of the hardware device. The processor may include at least one of a central processing unit (CPU) and a graphics processing unit (GPU). The specific test goal is to obtain the number of floating-point operations per second of the device to be deployed.

[0093] Based on this, when obtaining computing power assessment results, the server can perform data processing tests using matrix multiplication. Matrix multiplication is a common mathematical operation used in fields such as image processing, signal processing, and machine learning. Because the computational complexity of matrix multiplication increases significantly with the size of the matrix, it can be used to test the CPU's floating-point performance.

[0094] For example, as shown in Figure 4, when performing a data processing test on a device to be deployed, two random matrices of different sizes can be generated. The sizes of these two random matrices can be preset or randomly generated, and this application does not impose any restrictions on this. The server only needs to record the matrix sizes corresponding to the two random matrices.

[0095] Assuming that the two random matrices are random matrix A and random matrix B, after obtaining the two matrices, the server can perform multiplication operations on random matrix A and random matrix B based on the hardware environment of the device to be deployed, thereby obtaining matrix C.

[0096] It should be noted that there are two ways to perform matrix operations based on the hardware environment of the deployed device. One approach is for the server to directly instruct the deployed device to perform matrix operations, thereby obtaining the corresponding matrix C and collecting relevant computational data. This allows the server to fully collect the relevant computational performance of the deployed device, resulting in a more realistic computing power assessment.

[0097] Alternatively, the server can simulate the device within the server based on its hardware parameters, performing matrix multiplications directly on the server to obtain relevant computational data. This eliminates the need for data processing tests on a specific device. The server can directly obtain computing power assessment results for a relatively universal set of devices, without being affected by the flaws of a single device affecting the overall evaluation of that type of device.

[0098] While the server obtains the generated matrix C, it also needs to collect the calculation time of the matrix multiplication, so that the server can calculate the number of floating-point operations per second of the device to be deployed based on the size of the random matrix A and the random matrix B, and the total test processing time for the device to be deployed to complete the matrix calculation; then the server can use the number of floating-point operations per second as the computing power evaluation result corresponding to the device to be deployed.

[0099] Optionally, in order to avoid the possible particularities of the randomly generated matrix, when performing data processing tests on the device to be deployed, the number of tests can be appropriately increased, and the results of multiple tests can be arithmetic averaged or otherwise statistically processed, and the obtained statistical results can be used as the computing power evaluation results corresponding to the device to be deployed. In this way, the adverse effects caused by accidental results are avoided.

[0100] By testing the computing power of the device to be deployed, after obtaining the computing power evaluation result of the device to be deployed, the server can limit the quantization direction of the quantization model based on the computing power evaluation result.

[0101] S202: Based on the computing power evaluation result, at least one target operator is selected from the operators included in the model to be quantified, and the data processing time of the target operator on the device to be deployed reaches a set threshold.

[0102] The various operation units included in the model to be quantized are the operators mentioned in S202. Therefore, for a model, different operators require different amounts of computation to complete their operation processes.

[0103] Based on this, for the GFLOPS that the device to be deployed can provide, different operators may take different amounts of time to complete data processing on the same device to be deployed, and the cumulative data processing time consumed by these operators is the processing time required for the entire quantized model to complete an image segmentation process on the device to be deployed.

[0104] Therefore, when the total data processing time of a model to be quantified needs to be greater than the total expected processing time set by the developer, the server needs to perform targeted processing on the different operators contained in the model to be quantified.

[0105] For example, as shown in FIG5 , the server may separately count the data processing time corresponding to each operator when completing data processing on the device to be deployed, and then select target operators whose processing time exceeds a set threshold from a number of data processing times, and perform quantitative processing on these target operators.

[0106] For another example, as shown in FIG6 , the server can set an expected processing time for each operator included in the model to be quantified. In this way, when the data processing time of each operator on the device to be deployed exceeds the corresponding set threshold, the server can select the target operators whose data processing time exceeds the corresponding expected processing time from these operators, and perform quantification processing on these target operators.

[0107] Specifically, the server can implement the target operator screening by executing the screening method shown in Figure 7. Referring to Figure 7, which is a flow chart of a target operator screening method provided in an embodiment of the present application, as shown in Figure 7, the specific implementation steps of the method are as follows:

[0108] S701: Based on the computing power evaluation result, obtain the data processing time corresponding to each operator included in the model to be quantified when performing data processing in the device to be deployed.

[0109] The server can directly calculate the data processing time required for each operator based on the computing power evaluation results of the device to be deployed (i.e., the number of floating-point operations per second) and the amount of computation required for each operator included in the model to be quantified to complete the computation. Alternatively, the server can also perform simulation operations within the server based on the computing power evaluation results of the device to be deployed and directly collect the data processing time corresponding to each operator included in the model to be quantified. This application does not impose any restrictions on this.

[0110] S702: Based on the processing time of each data and a set threshold, at least one target operator is selected from each operator.

[0111] Among them, the configuration of setting the threshold can be configured by the developer for each operator individually based on his / her pre-expected processing time, or the server can directly allocate it to each operator in the model to be quantified based on the total expected processing time expected by the developer. This application does not impose any restrictions on this.

[0112] After obtaining the data processing time and the corresponding set threshold corresponding to each operator, the server can filter out at least one target operator whose data processing time exceeds the corresponding set threshold from each operator.

[0113] Through the above method, operators with excessively long data processing times can be identified from the model to be quantized as target operators. In this way, by quantizing these operators, the computational time of the model to be quantized on the device to be deployed can be effectively reduced, thereby improving quantization efficiency.

[0114] After completing the screening of target operators, the server can perform the following operations on these target operators:

[0115] S203: Based on the obtained difference between the data processing time of each of the at least one target operator and the corresponding expected processing time, quantize the at least one target operator to obtain a corresponding quantized operator, where the data processing time of the quantized operator on the device to be deployed is less than the expected processing time.

[0116] Quantization is a technique that converts the parameters of the target operator in the model to be quantized (such as convolution kernels, activation functions, pooling layers, etc.) into a representation that is easier to perform numerical calculations. Through quantization, the computational efficiency and accuracy of the target operator in the model to be quantized can be improved. Specifically, quantization quantizes the parameters of the target operator and converts them into 8-bit unsigned integers (such as 0 or 1) to facilitate subsequent calculations and processing.

[0117] It should be noted that the value of the expected processing time proposed in S203 can be equal to the value of the set threshold in S202. In this way, the quantization operator obtained after the quantization process can avoid being re-determined as the target operator in the subsequent re-screening process. Alternatively, the value corresponding to the expected processing time can also be less than the value corresponding to the set threshold. In this way, the time taken by the quantization operator obtained after the quantization process to complete the data processing will be shorter, and the total time for the entire model to complete the image segmentation process can also be shorter, which further improves the deployment effect of the image segmentation model on low-cost hardware equipment.

[0118] Optionally, when performing quantization processing on each target operator, the server may also implement the quantization processing effect in the following manner.

[0119] Specifically, the server may obtain the quantization parameter corresponding to the at least one target operator based on the obtained difference between the data processing duration of the at least one target operator and the corresponding expected processing duration.

[0120] Among them, the quantization parameters corresponding to the target operator may include parameters such as quantization factors or bias values. In this way, the quantization processing performed on the target operator can be to adjust some parameters in the target operator so that the time it takes for the target operator to complete data processing is less than the expected processing time.

[0121] Alternatively, the quantization processing of the target operator may be to replace the target operator with an operator with a different structure but the same function, so that the time taken by the replaced operator to complete the operation processed by the target operator meets the expected processing time requirement.

[0122] Then, based on these quantization parameters and the accuracy corresponding to at least one target operator, quantization processing is performed on at least one target operator to obtain the corresponding quantization accuracy; the error between the accuracy corresponding to the quantization operator and the accuracy corresponding to the target operator is less than a preset range.

[0123] When quantizing the target operator, different quantization methods can be used, such as linear quantization, nonlinear quantization, or symmetric quantization. This application does not impose any restrictions on this. It only requires that the error between the precision of the quantization operator and the precision of the target operator be less than a preset range during the quantization process. The preset range can be determined directly by the developer or configured based on parameters obtained by the server during training.

[0124] The quantization parameters determined according to the need to shorten the duration can be used to perform targeted quantization processing on the target operator with shortened duration, while taking into account the calculation accuracy of the target operator during quantization to achieve high-quality quantization.

[0125] S204: Based on at least one quantization operator, convert the model to be quantized into a corresponding target model.

[0126] After completing the acquisition of the above-mentioned quantization operator, the server can perform stream processing on the model to be quantized according to the acquired quantization operator, thereby converting the model to be quantized into the corresponding target model.

[0127] The target model achieved through conversion can be deployed to the device to be deployed.

[0128] In this solution, computing power tests are performed on the devices to be deployed to obtain corresponding computing power evaluation results. Based on the computing power evaluation results, the running status of the model to be quantified on the devices to be deployed is determined. Thus, each operator in the model to be quantified is quantified in a targeted manner to obtain the corresponding target model, thereby reducing its running time on the devices to be deployed, improving its deployment effect on the devices to be deployed, and reducing the difficulty of popularizing the target model.

[0129] Optionally, when converting the model to be quantized into a corresponding target model based on a quantization operator, the server may specifically perform the following operations to further reduce the processing time required for the target model to perform image segmentation on the device to be deployed.

[0130] Referring to FIG8 , which is a flow chart of a method for adjusting model accuracy provided in an embodiment of the present application, as shown in FIG8 , the specific implementation steps of the method are as follows:

[0131] S801: Construct a corresponding intermediate model based on at least one quantization operator and other operators in the model to be quantized except for at least one target operator.

[0132] After the server completes the quantization processing of the target operator, the server can construct the obtained quantization operators and the quanta originally included in the quantized model whose data processing time does not exceed the corresponding set threshold into an intermediate model.

[0133] In order to further reduce the time it takes for the target model to process image segmentation on the device to be deployed, the server can also reduce the model accuracy corresponding to the model (such as reducing the accuracy of the data processed by the model) to reduce the number of operations when the model performs image segmentation, thereby reducing the data processing time of the model.

[0134] S802: Determine the target accuracy corresponding to the target model based on the computing power evaluation result and the total expected processing time corresponding to the target model.

[0135] S803: Based on the target accuracy, adjust the model accuracy of the intermediate model to obtain the corresponding target model.

[0136] Specifically, as shown in FIG9 , the server needs to first determine the model accuracy achieved by the expected target model, and this determination process can be determined based on the computing power evaluation results corresponding to the device to be deployed and the total expected processing time corresponding to the target model.

[0137] For example, to determine whether the computing performance of the device to be deployed is sufficiently insufficient that, if the total data processing time of the target model is to reach the corresponding total expected processing time, the original possible floating point (FP) 32 precision needs to be converted to integer (INT) 8 precision. For another example, if it is determined that the computing power of the device to be deployed is sufficient to complete high-precision data processing within the total expected processing time, the server can convert the original FP32 precision model precision to FP16 precision. In this way, while retaining a certain degree of computing accuracy, the computing performance of the device to be deployed can be fully utilized, reducing the possibility of performance idleness.

[0138] Optionally, after completing the accuracy adjustment of the intermediate model, the server can also perform a model training based on the model to be calibrated after the accuracy adjustment, so as to take into account the accuracy and time consumption of the target model when performing image segmentation on the device to be deployed.

[0139] 10A is a flowchart of a method for training a model to be calibrated provided in an embodiment of the present application. As shown in FIG10A , the specific implementation steps of the method are as follows:

[0140] S1001: Acquire a sample set, wherein each sample in the sample set includes a sample segmentation image and a corresponding sample segmentation result.

[0141] The sample segmentation results corresponding to the sample segmented images in the sample set can be the image segmentation results obtained after image segmentation processing by the model to be quantized. In this way, in subsequent training, using the sample segmentation results as labels for training can maximize the guarantee that the image segmentation results obtained by the trained model to be calibrated can maintain the correspondence with the segmentation results of the original model to be quantized.

[0142] S1002: Setting the model to be calibrated in an operating environment with the same hardware parameters as the device to be deployed, obtaining a test segmentation result after the model to be calibrated performs image segmentation processing on the sample segmented image, and the image segmentation time taken by the model to be calibrated to complete the image segmentation processing.

[0143] Before training the model to be calibrated, it must first be set up in an operating environment with the same hardware parameters as the device to be deployed. To ensure the convergence efficiency of model training, the server can create a simulation environment that is identical to the operating environment of the device to be deployed. The model to be calibrated is then trained in this simulation environment, allowing the model to segment the sample segmentation image, obtain the corresponding test segmentation results, and simultaneously obtain the image segmentation time corresponding to the image segmentation process completed by the model to be calibrated.

[0144] S1003: Based on the error between the test segmentation result and the sample segmentation result, and the error between the image segmentation time and the total expected processing time, adjust the model parameters of the model to be calibrated to obtain the corresponding target model.

[0145] As shown in FIG10B , during a round of training, the server can adjust the model parameters of the model to be calibrated based on the loss value between the test segmentation result and the sample segmentation result corresponding to a certain sample segmentation image obtained, as well as the error between the image segmentation time consumed by the model to be calibrated when processing the sample segmentation image and the total expected processing time corresponding to the model to be calibrated, thereby completing a round of training for the model to be calibrated.

[0146] In this way, after multiple rounds of training as described above, the server can obtain a target model in which the overall model can take into account both image segmentation accuracy and image segmentation efficiency.

[0147] The above describes the target model acquisition process. Once the server has acquired the target model, it can use it to deploy it on the device to be deployed and perform image processing. For example, the server can deploy the target model in a VR device specifically for iris segmentation during iris recognition. Alternatively, the server can deploy it in a low-cost mobile phone to perform facial image segmentation during face recognition.

[0148] 11 is a flowchart of an image segmentation method provided in an embodiment of the present application. The specific implementation steps of the method are as follows:

[0149] S1101: Setting the target model in an operating environment with the same hardware parameters as the device to be deployed.

[0150] It should be noted that when deploying the target model, its deployment object is not limited to one device of the object to be deployed, but can be deployed in other devices with the same hardware parameters as the object to be deployed, or directly deployed in a virtual machine. This application does not impose any restrictions on this.

[0151] S1102: After the server completes the deployment of the target model, it can perform image segmentation processing on the iris image to be segmented based on at least one quantization operator in the target model and other operators in the model to be quantized except for the at least one target operator to obtain a corresponding iris image segmentation result.

[0152] As shown in Figure 12, after the deployment of the target model is completed, when the target model performs segmentation processing on the iris image, it can perform image segmentation processing on the iris image to be segmented based on the quantization operator after quantization processing contained in itself, as well as other operators other than the target operator that do not need to be quantized, thereby obtaining the corresponding iris image segmentation result.

[0153] In this way, the processing time of performing iris image segmentation on the device to be deployed is reduced, the implementation effect of iris recognition on low-cost devices is ensured, and iris recognition technology can be more widely applied and promoted on low-cost hardware devices.

[0154] In order to further reduce the processing time of iris image segmentation, the accuracy of the target model can also be set to a preset accuracy. At this time, the preset accuracy can be the INT8 accuracy or FP16 accuracy as mentioned above. In this way, the processing time required for the target model of this accuracy during image segmentation can be further reduced.

[0155] In order to match the accuracy of such a model, the image input to the target model can be adaptively adjusted in advance to ensure a match between the data input to the model and the accuracy of the model.

[0156] 13A is a flowchart of an image segmentation method provided in an embodiment of the present application. As shown in FIG13A , the specific implementation steps of the method are as follows:

[0157] S1301: Based on a preset accuracy, adjust the image accuracy corresponding to the iris image to be segmented, and obtain a corresponding intermediate image, where the image accuracy of the intermediate image corresponds to the preset accuracy.

[0158] The preset accuracy is the model accuracy corresponding to the target model. Therefore, when the preset accuracy is INT8, the server needs to adjust the image accuracy of the iris image to be segmented from the possible FP32 accuracy or FP16 accuracy to the corresponding INT8 accuracy. Only in this way can it be input into the target model for image segmentation processing.

[0159] S1302: Based on at least one quantization operator in the target model and other operators except the at least one target operator in the model to be quantized, perform image segmentation processing on the intermediate image to obtain a corresponding intermediate segmentation result.

[0160] At this point, after the target model with the preset accuracy performs image segmentation on the iris image to be segmented with the preset accuracy, the intermediate segmentation result obtained also has the preset accuracy. However, due to the mismatch in image accuracy, this intermediate segmentation result with the preset accuracy cannot be directly used as the image segmentation result corresponding to the original, high-accuracy iris image to be segmented. Therefore, the server must perform the following operations before outputting the final iris image segmentation result.

[0161] S1303: Based on the image accuracy corresponding to the iris image to be segmented, adjust the accuracy of the intermediate segmentation result to obtain the corresponding iris image segmentation result; wherein the image accuracy corresponding to the iris image segmentation result is the same as the image accuracy corresponding to the iris image to be segmented.

[0162] After completing the low-precision segmentation of the image to be segmented using the target model, the server can also dequantize the obtained low-precision intermediate segmentation results. For example, it can dequantize the low-precision INT8 precision results to FP32 precision or FP16 precision results. This can improve the accuracy and readability of the iris image segmentation results, and can meet the accuracy requirements of subsequent image processing processes, improving the feasibility of this solution.

[0163] As shown in Figure 13B, the iris image to be segmented with an image accuracy of FP32 can first undergo precision conversion to obtain an intermediate image with an image accuracy of INT8. The intermediate image is then input into the target model, and then the corresponding intermediate segmentation result with an accuracy of INT8 is obtained. Finally, the intermediate segmentation result with an accuracy of INT8 is converted into an iris image segmentation result with an accuracy of FP32.

[0164] The above introduces the process of quantizing the image model provided by the embodiments of the present application and some possible implementation methods. In order to further clarify the above various schemes, the following introduces the quantization process by combining the above methods into a possible solution.

[0165] 14 is a flowchart of a quantization method for an image segmentation model provided in an embodiment of the present application. The specific implementation steps of the method are as follows:

[0166] S1401: Acquire a model to be quantified and determine a device to be deployed on which the model to be quantified needs to be deployed.

[0167] S1402: Performing a data processing test on the device to be deployed based on multiple random data, and obtaining a test processing time required for the device to be deployed to complete the data processing test.

[0168] S1403: Determine a computing power evaluation result of the device to be deployed based on the test processing time and the amount of random data.

[0169] S1404: Based on the computing power evaluation result, at least one target operator included in the model to be quantified is screened out, wherein the data processing time of the target operator on the device to be deployed reaches a set threshold.

[0170] S1405: Based on the obtained difference between the data processing time of each of the at least one target operator and the corresponding expected processing time, quantize the at least one target operator to obtain a corresponding quantized operator, where the data processing time of the quantized operator on the device to be deployed is less than the expected processing time.

[0171] S1406: Construct a corresponding intermediate model based on at least one quantization operator and other operators in the model to be quantized except for at least one target operator.

[0172] S1407: Determine the target accuracy corresponding to the target model based on the computing power evaluation result and the total expected processing time corresponding to the target model.

[0173] S1408: Based on the target accuracy, adjust the model accuracy of the intermediate model to obtain the corresponding model to be calibrated.

[0174] S1409: Acquire a sample set, where each sample in the sample set includes a sample segmentation image and a corresponding sample segmentation result.

[0175] S1410: Setting the model to be calibrated in an operating environment with the same hardware parameters as the device to be deployed, obtaining a test segmentation result after the model to be calibrated performs image segmentation processing on the sample segmented image, and the image segmentation time it takes for the model to be calibrated to complete the image segmentation processing.

[0176] S1411: Based on the error between the test segmentation result and the sample segmentation result, and the error between the image segmentation time and the total expected processing time, adjust the model parameters of the model to be calibrated to obtain the corresponding target model.

[0177] As shown in FIG15 , the acquisition of the target model is completed, and the server may also set the target model in the device to be deployed to complete the image segmentation process of the corresponding iris image.

[0178] Based on the same inventive concept, an embodiment of the present application also provides a quantization device for an image segmentation model. See Figure 16, which is a structural schematic diagram of a quantization device for an image segmentation model provided in an embodiment of the present application. The device can be the above-mentioned terminal device or server or a chip or integrated circuit therein, etc. The device includes modules / units / technical means for executing the method executed by the terminal device or server in the above-mentioned method embodiment.

[0179] Exemplarily, the apparatus 1600 includes:

[0180] Acquisition module 1601 is used to obtain the computing power evaluation results of the device to be deployed and the model to be quantized; wherein the computing power evaluation results are used to indicate the computing performance of the device to be deployed, and the model to be quantized is used to implement image segmentation;

[0181] A screening module 1602 is configured to select, based on the computing power evaluation result, at least one target operator from among the operators included in the model to be quantified, where the data processing time of the target operator on the device to be deployed reaches a set threshold;

[0182] Quantization module 1603 is used to quantize at least one target operator based on the difference between the data processing time of each target operator and the corresponding expected processing time, and obtain a corresponding quantization operator, where the data processing time of the quantization operator on the device to be deployed is less than the expected processing time; based on the at least one quantization operator, convert the model to be quantized into a corresponding target model, and the target model is used to be deployed on the device to be deployed.

[0183] In a possible implementation, when the quantization module 1603 is configured to convert the to-be-quantized model into a corresponding target model based on at least one quantization operator, it is specifically configured to:

[0184] Constructing a corresponding intermediate model based on at least one quantization operator and other operators in the model to be quantized except for at least one target operator;

[0185] Determine the target accuracy of the target model based on the computing power evaluation results and the total expected processing time of the target model.

[0186] Based on the target accuracy, the model accuracy of the intermediate model is adjusted to obtain the corresponding target model.

[0187] In one possible implementation, the quantization module 1603 is configured to adjust the model accuracy of the intermediate model based on the target accuracy, and to obtain the corresponding target model by:

[0188] Based on the target accuracy, the model accuracy of the intermediate model is adjusted to obtain the corresponding model to be calibrated;

[0189] Obtain a sample set, where each sample in the sample set includes a sample segmentation image and a corresponding sample segmentation result;

[0190] The model to be calibrated is set in an operating environment with the same hardware parameters as the device to be deployed, and the test segmentation results after the model to be calibrated performs image segmentation processing on the sample segmented image, as well as the image segmentation time it takes for the model to be calibrated to complete the image segmentation processing;

[0191] Based on the error between the test segmentation result and the sample segmentation result, as well as the error between the image segmentation time and the total expected processing time, the model parameters of the calibration model are adjusted to obtain the corresponding target model.

[0192] In one possible implementation, when the acquisition module 1601 is used to obtain the computing power evaluation result of the device to be deployed, it is specifically used to:

[0193] Based on multiple random data, perform data processing tests on the devices to be deployed, and obtain the test processing time required for the data processing tests on the devices to be deployed;

[0194] Based on the test processing time and the data volume corresponding to multiple random data, obtain the computing power evaluation results corresponding to the device to be deployed.

[0195] In one possible implementation, the screening module 1602 is configured to select at least one target operator from the operators included in the model to be quantified based on the computing power evaluation result, specifically to:

[0196] Based on the computing power evaluation results, obtain the data processing time corresponding to each operator included in the model to be quantified when processing data in the device to be deployed;

[0197] Based on the data processing time and the set threshold, at least one target operator is selected from each operator.

[0198] In one possible implementation, the quantization module 1603 is configured to perform quantization processing on at least one target operator based on the difference between the obtained data processing time of each of the at least one target operator and the corresponding expected processing time, and to obtain the corresponding quantization operator by:

[0199] Obtaining a quantization parameter corresponding to the at least one target operator based on a difference between the obtained data processing time of each of the at least one target operator and the corresponding expected processing time;

[0200] Based on the quantization parameter and the precision corresponding to at least one target operator, quantize the at least one target operator to obtain the corresponding quantization operator; wherein the error between the precision corresponding to the quantization operator and the precision corresponding to the target operator is less than a preset range.

[0201] In a possible implementation, the apparatus 1600 further includes a processing module 1604. After acquiring the target model, the processing module 1604 is configured to:

[0202] Set the target model to run in an environment with the same hardware parameters as the device to be deployed;

[0203] Based on at least one quantization operator in the target model and other operators except the at least one target operator in the model to be quantized, image segmentation processing is performed on the iris image to be segmented to obtain a corresponding iris image segmentation result.

[0204] In a possible implementation, the model accuracy corresponding to the target model is a preset accuracy;

[0205] The processing module 1604 is configured to perform image segmentation processing on the iris image to be segmented based on at least one quantization operator in the target model and other operators other than the at least one target operator in the model to be quantized, and to obtain a corresponding iris image segmentation result, specifically for:

[0206] Based on the preset accuracy, adjusting the image accuracy corresponding to the iris image to be segmented to obtain a corresponding intermediate image, wherein the image accuracy of the intermediate image corresponds to the preset accuracy;

[0207] Performing image segmentation processing on the intermediate image based on at least one quantization operator in the target model and other operators in the model to be quantized except the at least one target operator to obtain a corresponding intermediate segmentation result;

[0208] Based on the image accuracy corresponding to the iris image to be segmented, the accuracy of the intermediate segmentation result is adjusted to obtain the corresponding iris image segmentation result; wherein the image accuracy corresponding to the iris image segmentation result is the same as the image accuracy corresponding to the iris image to be segmented.

[0209] As an embodiment, the device discussed in FIG16 can be used to execute the method described in the embodiment shown in FIG2 . Therefore, for the functions that can be implemented by the various functional modules of the device, please refer to the description of the embodiment shown in FIG2 , and no further details will be given here.

[0210] Memory 1701 is used to store computer programs executed by processor 1702. Memory 1701 may mainly include a program storage area and a data storage area. The program storage area may store an operating system and programs required for running instant messaging functions, while the data storage area may store various instant messaging messages and operating instruction sets.

[0211] Memory 1701 may be a volatile memory, such as random-access memory (RAM); a non-volatile memory, such as read-only memory, flash memory, a hard disk drive (HDD), or a solid-state drive (SSD); or any other medium capable of carrying or storing a desired computer program in the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory 1701 may be a combination of the aforementioned memories.

[0212] The processor 1702 may include one or more central processing units (CPUs) or digital processing units, etc. The processor 1702 is configured to implement the quantization method of the above-mentioned image segmentation model when calling the computer program stored in the memory 1701 .

[0213] The communication module 1703 is used to communicate with terminal devices and other servers.

[0214] The specific connection medium between the memory 1701, communication module 1703, and processor 1702 is not limited in the embodiments of the present application. In Figure 17, the memory 1701 and processor 1702 are connected via bus 1704. Bus 1704 is depicted as a bold line in Figure 17. The connection methods between other components are merely schematic and are not intended to be limiting. Bus 1704 can be divided into an address bus, a data bus, a control bus, and the like. For ease of description, Figure 17 depicts only one bold line, but does not represent only one bus or one type of bus.

[0215] The memory 1701 stores a computer storage medium, which stores computer-executable instructions for implementing the image segmentation model quantization method of the embodiment of the present application. The processor 1702 is used to execute the above-mentioned image segmentation model quantization method.

[0216] In another embodiment, the electronic device may be another electronic device, such as the terminal device 101 shown in FIG1B . In this embodiment, the structure of the electronic device may be as shown in FIG18 , including: a communication component 1810, a memory 1818, a display unit 1830, a camera 1840, a sensor 1850, an audio circuit 1860, a Bluetooth module 1870, a processor 1880, and other components.

[0217] The communication component 1810 is used to communicate with the server. In some embodiments, it may include a wireless fidelity (WiFi) module. The WiFi module is a short-range wireless transmission technology. The electronic device can help the object send and receive information through the WiFi module.

[0218] The memory 1818 may be used to store software programs and data. The processor 1880 executes the software programs or data stored in the memory 1818 to perform various functions of the terminal device 101 and process data.

[0219] The display unit 1830 may also be used to display information input by or provided to an object, as well as a graphical user interface (GUI) of various menus of the terminal device 101 .

[0220] The physical terminal device may further include at least one sensor 1850, such as an accelerometer 1851, a distance sensor 1852, a fingerprint sensor 1853, and a temperature sensor 1854. The terminal device may also be equipped with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, a light sensor, and a motion sensor.

[0221] The audio circuit 1860 , the speaker 1861 , and the microphone 1862 may provide an audio interface between the object and the terminal device 101 .

[0222] Processor 1880 is the control center of the physical terminal device, connecting the various components of the entire terminal using various interfaces and lines. It executes various terminal device functions and processes data by running or executing software programs stored in memory 1818 and accessing data stored in memory 1818. In some embodiments, processor 1880 may include one or more processing units. Processor 1880 may also integrate an application processor and a baseband processor, where the application processor primarily processes the operating system, user interface, and application programs, while the baseband processor primarily handles wireless communications. It is understood that the baseband processor may not be integrated into processor 1880. In this application, processor 1880 can run the operating system, application programs, user interface display and touch response, as well as the methods of the embodiments of this application. In addition, processor 1880 is coupled to display unit 1830.

[0223] In addition, it should be noted that in the specific implementation of this application, object data related to image segmentation models, etc. is involved. When the above embodiments of this application are applied to specific products or technologies, it is necessary to obtain the object's permission or consent, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0224] In addition, an embodiment of the present application further provides a storage medium, which is used to store a computer program, and the computer program is used to execute the method provided by the above embodiment.

[0225] An embodiment of the present application further provides a computer program product including a computer program, which, when executed on a computer, enables the computer to execute the method provided in the above embodiment.

[0226] It should be noted that although several units or subunits of the device are mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, depending on the embodiment of the application, the features and functions of two or more units described above can be embodied in a single unit. Conversely, the features and functions of a single unit described above can be further divided and embodied by multiple units.

[0227] Furthermore, although the operations of the method of the present application are described in a particular order in the accompanying drawings, this does not require or imply that the operations must be performed in this particular order, or that all illustrated operations must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0228] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0229] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each flow and / or box in the flow chart and / or block diagram, as well as the combination of the flow chart and / or box in the flow chart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more flow charts and / or one or more boxes in the block diagram.

[0230] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0231] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0232] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A method for quantizing an image segmentation model, the method being executed by a computer device, the method comprising: Obtaining a computing power evaluation result of the device to be deployed and a model to be quantified; wherein the computing power evaluation result is used to indicate the computing performance of the device to be deployed, and the model to be quantified is used to implement image segmentation; Based on the computing power evaluation result, at least one target operator is selected from the operators included in the model to be quantified, and the data processing time of the target operator on the device to be deployed reaches a set threshold; Based on the obtained difference between the data processing time of each of the at least one target operator and the corresponding expected processing time, quantizing the at least one target operator to obtain a corresponding quantized operator, wherein the data processing time of the quantized operator on the device to be deployed is less than the expected processing time; Based on the at least one quantization operator, the model to be quantized is converted into a corresponding target model, and the target model is used to be deployed to the device to be deployed.

2. The method according to claim 1, wherein converting the to-be-quantized model into a corresponding target model based on the at least one quantization operator comprises: Based on the at least one quantization operator and other operators in the model to be quantized except the at least one target operator, construct a corresponding intermediate model; Determine a target accuracy corresponding to the target model based on the computing power evaluation result and the total expected processing time corresponding to the target model; Based on the target accuracy, the model accuracy of the intermediate model is adjusted to obtain a corresponding target model.

3. The method according to claim 2, wherein the model accuracy of the intermediate model is adjusted based on the target accuracy to obtain the corresponding target model, comprising: Based on the target accuracy, adjusting the model accuracy of the intermediate model to obtain a corresponding model to be calibrated; Acquire a sample set, each sample in the sample set includes a sample segmentation image and a corresponding sample segmentation result; The model to be calibrated is set in an operating environment with the same hardware parameters as the device to be deployed, and a test segmentation result after the model to be calibrated performs image segmentation processing on the sample segmented image, and an image segmentation time taken by the model to be calibrated to complete the image segmentation processing; Based on the error between the test segmentation result and the sample segmentation result, and the error between the image segmentation time and the total expected processing time, the model parameters of the model to be calibrated are adjusted to obtain the corresponding target model.

4. The method according to any one of claims 1 to 3, wherein obtaining the computing power evaluation result of the device to be deployed comprises: Based on the multiple random data, a data processing test is performed on the device to be deployed, and a test processing time required for the device to be deployed to perform the data processing test is obtained; Based on the test processing time and the data volume corresponding to the multiple random data, a computing power evaluation result corresponding to the device to be deployed is obtained.

5. The method according to claim 4, wherein the step of selecting at least one target operator from the operators included in the model to be quantified based on the computing power evaluation result comprises: Based on the computing power evaluation result, the operations of each operator included in the to-be-quantified model in the to-be-deployed device are obtained. The corresponding data processing time when processing row data; Based on the data processing time and the set threshold, the at least one target operator is selected from the operators.

6. The method according to claim 1, wherein the step of performing quantization processing on the at least one target operator based on the obtained difference between the data processing time of each of the at least one target operator and the corresponding expected processing time to obtain the corresponding quantized operator comprises: Based on the obtained difference between the data processing time of each of the at least one target operator and the corresponding expected processing time, obtaining a quantization parameter corresponding to the at least one target operator; Based on the quantization parameter and the accuracy corresponding to the at least one target operator, the at least one target operator is quantized to obtain a corresponding quantization operator; wherein an error between the accuracy corresponding to the quantization operator and the accuracy corresponding to the target operator is less than a preset range.

7. The method according to any one of claims 1 to 6, after obtaining the target model, the method further comprises: Setting the target model in an operating environment with the same hardware parameters as the device to be deployed; Based on at least one quantization operator in the target model and other operators in the model to be quantized except the at least one target operator, image segmentation processing is performed on the iris image to be segmented to obtain a corresponding iris image segmentation result.

8. The method according to claim 7, wherein the model accuracy corresponding to the target model is a preset accuracy; Then, performing iris image segmentation processing on the image to be segmented based on at least one quantization operator in the target model and other operators in the model to be quantized except the at least one target operator to obtain a corresponding iris image segmentation result includes: Based on the preset accuracy, adjusting the image accuracy corresponding to the iris image to be segmented to obtain a corresponding intermediate image, wherein the image accuracy of the intermediate image corresponds to the preset accuracy; Based on at least one quantization operator in the target model and other operators in the model to be quantized except the at least one target operator, performing image segmentation processing on the intermediate image to obtain a corresponding intermediate segmentation result; Based on the image accuracy corresponding to the image to be segmented, the accuracy of the intermediate segmentation result is adjusted to obtain a corresponding iris image segmentation result; wherein the image accuracy corresponding to the iris image segmentation result is the same as the image accuracy corresponding to the iris image to be segmented.

9. A quantization device for an image segmentation model, the device comprising: An acquisition module, used to acquire a computing power evaluation result of a device to be deployed and a model to be quantified; wherein the computing power evaluation result is used to indicate the computing performance of the device to be deployed, and the model to be quantified is used to implement image segmentation; A screening module, configured to select, based on the computing power evaluation result, at least one target operator from among the operators included in the model to be quantified, wherein the data processing time of the target operator on the device to be deployed reaches a set threshold; A quantization module is used to quantize at least one target operator based on the difference between the data processing time of each of the at least one target operator and the corresponding expected processing time, and obtain a corresponding quantization operator, wherein the data processing time of the quantization operator on the device to be deployed is less than the expected processing time; based on the at least one quantization operator, the model to be quantized is converted into a corresponding target model, and the target model is used to be deployed to the device to be deployed.

10. An electronic device comprising a processor and a memory, wherein: The memory stores program codes, and when the program codes are executed by the processor, the processor is enabled to perform the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium for storing a computer program, wherein when the computer program is run on a computing device, the computer program is used to cause the computing device to execute the steps of the method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images