Quantization method, device, equipment, and computer program for image segmentation model

Abstract

The present application relates to the field of computer technology and provides a quantization method, device, equipment and computer storage medium for an image segmentation model, the method including the steps of: obtaining a computing power evaluation result and a quantization target model of a target device to be introduced, the computing power evaluation result being used to indicate the computing performance of the target device to be introduced, and the quantization target model being used to realize image segmentation; selecting at least one target operator from each operator included in the quantization target model based on the computing power evaluation result, wherein the target operator is selected when the data processing time length of the target device to be introduced reaches a set threshold; performing quantization processing on the at least one target operator based on the difference between each data processing time length of the acquired at least one target operator and a corresponding desired processing time length, to obtain a corresponding quantization operator, wherein the data processing time length of the target device to be introduced is shorter than the desired processing time length; and converting the quantization target model into a corresponding target model based on the at least one quantization operator.

Description

[Technical Field]

[0001] This application claims priority from a Chinese patent application bearing application number 2023109160698 and entitled "Image segmentation model quantization method, device, apparatus and computer storage medium" filed with the China Patent Office on July 25, 2023, the entire contents of which are incorporated herein by reference.

[0002] This application relates to the technical field of computers, and more particularly to quantization of image segmentation models. [Background technology]

[0003] With the continuous development of Internet technology, various image segmentation models have been able to achieve more precise and accurate image segmentation effects, but achieving such effects usually relies on high hardware costs, and directly applying these models to low-cost hardware devices makes it difficult to achieve the expected results.

[0004] For example, in some image recognition processes, an image must first be segmented, and then feature extraction must be performed on the segmented image to achieve image recognition. This process consumes excessive computing power, and in order to successfully achieve the above process, the hardware responsible for this process must have high computing power. This makes it difficult to complete high-precision image recognition with low-cost hardware equipment, reducing the efficiency of its widespread application.

[0005] In particular, for iris recognition technology applied to virtual reality (VR) devices, most segmentation algorithms used in the image segmentation step of the iris recognition process are implemented in embedded systems, so implementing iris segmentation usually requires high hardware costs and complex software design. Therefore, when it is implemented in low-cost hardware devices, it takes a long time for the device to perform the iris segmentation step, making it difficult to achieve the desired real-time effect. This makes it difficult to implement iris recognition technology in low-cost hardware devices, hindering the need for its large-scale application.

[0006] Therefore, how to improve the effectiveness of introducing image segmentation models into low-cost hardware devices has become an issue that must be resolved urgently. Summary of the Invention [Problem to be solved by the invention]

[0007] The present application provides an image segmentation method, apparatus and device, computer storage medium and program product, to realize improving the implementation effect of image segmentation models in low-cost hardware devices. [Means for solving the problem]

[0008] According to a first aspect, the present application provides a method for quantizing an image segmentation model, the method comprising: a step of acquiring a computing power evaluation result and a quantization target model of the target device, the computing power evaluation result being used to indicate the computing performance of the target device, and the quantization target model being used to realize image segmentation; A step of selecting at least one target operator from each operator included in the quantization target model based on the calculation performance evaluation result, wherein the target operator is selected when the data processing time length in the target device reaches a set threshold; A step of performing a quantization process on at least one target operator based on a difference between the data processing time length of each of the acquired at least one target operator and a corresponding desired processing time length to acquire a corresponding quantization operator, wherein the data processing time length of the quantization operator in the target device is shorter than the desired processing time length; converting the quantized target model into a corresponding target model based on at least one quantization operator, the target model being used for deployment to the deployment target device.

[0009] According to a second aspect, the present application provides an apparatus for quantizing an image segmentation model, the apparatus comprising: an acquisition module for acquiring a computing power evaluation result and a quantization target model of a target device for introduction, the computing power evaluation result being used to indicate the computing performance of the target device for introduction, and the quantization target model being used to realize image segmentation; a selection module for selecting at least one target operator from among the operators included in the model to be quantized based on the calculation performance evaluation result, the target operator being one whose data processing time length in the device to be introduced reaches a set threshold; a quantization module for performing a quantization process on at least one target operator based on a difference between the data processing time length of each of the acquired at least one target operator and a corresponding desired processing time length to obtain a corresponding quantization operator, wherein the data processing time length in the device to be introduced is shorter than the desired processing time length, and the quantization module converts a quantization target model into a corresponding target model based on the at least one quantization operator, and the target model is used for introduction into the device to be introduced.

[0010] According to a third aspect, the present application provides an electronic device including a processor and a memory, wherein the memory stores program code that, when executed by the processor, causes the processor to perform steps of any of the above-described image segmentation model quantization methods.

[0011] According to a fourth aspect, the present application further provides a computer-readable storage medium, used to store a computer program, which, when executed on an electronic device, causes the electronic device to perform steps of any of the above-described image segmentation model quantization methods.

[0012] According to a fifth aspect, the present application further provides a computer program product, comprising a computer program that, when executed by a processor, implements the steps of any of the above-described image segmentation model quantization methods.

[0013] The beneficial effects of the present invention are as follows:

[0014] The present application provides a method, apparatus, device, and storage medium for quantizing an image segmentation model, the method including: obtaining a computing power evaluation result of a target device; determining the execution status of the quantization target model on the target device according to the computing power evaluation result; appropriately quantizing each operator in the quantization target model; obtaining a quantization operator with a data processing time that meets the required data processing time; using the operators as the quantization granularity, reducing the number of operations when quantizing the model and improving the corresponding quantization efficiency; and constructing a corresponding target model according to the quantization operators; ensuring that when the target model is executed on the target device, the processing time of the image segmentation task can be effectively reduced to the total desired processing time, thereby improving the implementation efficiency of low-cost hardware and effectively increasing the popularity of the image segmentation model in low-cost devices.

[0015] In the process of quantizing the model, the accuracy of the model can be adjusted based on the predetermined accuracy requirements, so as to match the computing power of the target equipment, thereby improving the introduction effect of the target model in the target equipment.

[0016] Additional features and advantages of the present application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the present application. The objectives and other advantages of the present application will be realized and attained by the structure particularly pointed out in the written description, claims, and drawings. [Brief explanation of the drawings]

[0017] [Figure 1A] FIG. 1 is a schematic diagram of image division provided by an embodiment of the present application. [Figure 1B] 1 is a schematic diagram of a possible application scenario provided by an embodiment of the present application; [Figure 2] 1 is a flowchart of a quantization method for an image segmentation model provided by an embodiment of the present application; [Figure 3] 1 is a flowchart of a method for obtaining a computing power evaluation result provided by an embodiment of the present application; [Figure 4] FIG. 1 is a schematic diagram of obtaining a computing power evaluation result provided by an embodiment of the present application. [Figure 5] FIG. 1 is a schematic diagram of a target operator selection method provided by an embodiment of the present application; [Figure 6] FIG. 1 is a schematic diagram of another target operator selection method provided by an embodiment of the present application; [Figure 7] 1 is a flowchart of a target operator selection method provided by an embodiment of the present application; [Figure 8] 1 is a flowchart of a method for adjusting model accuracy provided by an embodiment of the present application; [Figure 9] FIG. 1 is a schematic diagram of a method for adjusting model accuracy provided by an embodiment of the present application. [Figure 10A] 1 is a flowchart of a method for training a calibration target model provided by an embodiment of the present application; [Figure 10B] FIG. 1 is a schematic diagram of training a calibration target model provided by an embodiment of the present application; [Figure 11] 1 is a flowchart of an image segmentation method provided by an embodiment of the present application; [Figure 12] FIG. 1 is a schematic diagram of an image segmentation method provided by an embodiment of the present application. [Figure 13A] 1 is a flowchart of an image segmentation method provided by an embodiment of the present application; [Figure 13B] FIG. 1 is a schematic diagram of an image segmentation method provided by an embodiment of the present application. [Figure 14] 1 is a flowchart of a quantization method for an image segmentation model provided by an embodiment of the present application; [Figure 15] FIG. 1 is a schematic diagram of a quantization method for an image segmentation model provided by an embodiment of the present application; [Figure 16] FIG. 2 is a structural schematic diagram of a quantization device for an image segmentation model provided by an embodiment of the present application; [Figure 17] 1 is a schematic diagram of one hardware configuration structure of an electronic device according to an embodiment of the present application. [Figure 18] FIG. 2 is a schematic diagram of a hardware configuration of another electronic device according to an embodiment of the present application. DETAILED DESCRIPTION OF THE INVENTION

[0018] In order to clarify the objectives, technical solutions, and advantages of the present application, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all of the embodiments. All other embodiments obtained based on the embodiments of the present application without the need for creative work by those skilled in the art should fall within the scope of protection of the present application. Unless contradictory, the embodiments and features of the embodiments of the present application may be arbitrarily combined with each other. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order.

[0019] In the following specific embodiments of the present application, when the examples of the present application are applied to specific products or technologies, relevant permissions or consents must be obtained for data related to sample images, etc., and the collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, when relevant data needs to be obtained, relevant volunteers can be recruited and the volunteers can sign relevant contracts authorizing the data, and the data of these volunteers can be used. Alternatively, the following embodiments can be implemented within the scope of an authorized organization, using data from internal members of the organization to perform relevant recognition of internal members. Alternatively, the relevant data used in the specific implementation can be simulation data, such as simulation data generated in a virtual scene.

[0020] In order to facilitate understanding of the technical solutions provided by the embodiments of the present application, some important terms used in the embodiments of the present application will be explained here.

[0021] Image segmentation: A typical computer vision problem involves taking some raw data (e.g., a planar image) as input and converting it into a mask with regions of interest to highlight. As shown in Figure 1A, when performing image segmentation on an original image a (upper part of Figure 1A) that has images of different categories, if we restrict the category that needs to be highlighted to that of vehicles, after processing with an image segmentation model, we obtain an image segmentation result that highlights the outlines of two vehicles in the panoramic image, as shown in segmentation result b (lower part of Figure 1A).

[0022] Operator (OP): An individual calculation unit of a deep learning algorithm. In a neural network model, an operator corresponds to the calculation logic of a layer. For example, a convolution layer may be an operator, and the weight summation process in a fully-connected layer (FC layer) may also be an operator.

[0023] Floating-point operations per second (GFLOPS): Also known as peak rate per second, it is the number of floating-point operations performed per second. Floating-point refers to numbers with decimals, and floating-point arithmetic is the four basic arithmetic operations with decimals. It is often used to measure computer speed and estimate computer performance, especially in the field of scientific computing, which uses a large number of floating-point operations.

[0024] Floating Point Operations (FLOPs): can be used to measure the complexity of a model / algorithm.

[0025] The design concept of the embodiment of the present invention will be briefly introduced below.

[0026] With the development of Internet technology, more and more network models can achieve more accurate technical effects, for example, image recognition models can more accurately recognize a target image from multiple images, and image segmentation models can more accurately achieve the effect of image segmentation. However, the realization of these technical effects currently still relies on high hardware costs, and it is difficult to achieve the expected effects when these network models are directly applied to low-cost hardware.

[0027] For example, in some image recognition processes, an image must first be segmented, and then feature extraction must be performed on the segmented image to achieve image recognition. This process consumes excessive computing power, and in order to successfully achieve this process, the hardware responsible for this process must have high computing power. This makes it difficult to achieve high-precision image recognition with low-cost hardware equipment, reducing the efficiency of its widespread application.

[0028] In particular, for iris recognition technology applied to virtual reality (VR) devices, most segmentation algorithms used in the image segmentation step of the iris recognition process are implemented in embedded systems, so implementing iris segmentation usually requires high hardware costs and complex software design. Therefore, when it is implemented in low-cost hardware devices, it takes a long time for the device to perform the iris segmentation step, making it difficult to achieve the desired real-time effect. This makes it difficult to implement iris recognition technology in low-cost hardware devices, hindering the need for its large-scale application.

[0029] In view of this, the embodiments of the present application provide a quantization method for an image segmentation model to improve the introduction effect of the image segmentation model in low-cost hardware devices.

[0030] This method first evaluates the computing power of a target device in which an image segmentation model is to be introduced, thereby obtaining a computing power evaluation result for indicating the computing performance of the target device, and then obtains a quantization target model for realizing image segmentation. Based on the computing power evaluation result for the target device, at least one target operator is selected from each operator included in the quantization target model, whose data processing time length on the target device reaches a set threshold. Based on the difference between the data processing time length of the target operator and the corresponding desired processing time length, a quantization process is performed on the target operator, thereby obtaining a corresponding quantization operator. The quantization target model is then converted into a corresponding target model in accordance with the quantization operator. After performing the quantization process on each operator in the quantization target model, the processing time length when the obtained target model performs image segmentation on the target device can meet the desired requirements, effectively improving the effectiveness of its introduction in low-cost hardware devices.

[0031] Furthermore, after performing the quantization process for each operator, a corresponding intermediate model can be constructed based on these quantization operators and other operators in the model to be quantized. Then, a target accuracy for the target model is determined according to the calculation performance evaluation result and the total desired processing time for the target model. In this way, the model accuracy for the intermediate model can be adjusted based on the target accuracy, and the model accuracy is made the same as the target accuracy, thereby obtaining a target model whose processing time for completing the image segmentation task on the target device satisfies the total desired processing time.

[0032] Furthermore, after completing the accuracy conversion of the intermediate model, the calibration target model after the accuracy conversion is completed can be set to an execution environment with the same hardware parameters as the equipment to be introduced, and model parameters can be adjusted for the calibration target model based on several sample segmentation images and sample segmentation results, so that the target model after parameter adjustment can achieve both accuracy and efficiency in image segmentation.

[0033] In the embodiments of the present application, the part that acquires the quantization target model and the quantization operator and adjusts the model parameters is related to artificial intelligence (AI), computer vision technology (CV), and machine learning (ML) technology, and is designed based on image segmentation technology and machine learning in artificial intelligence.

[0034] Artificial intelligence (AI) is the theory, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, expand, and enhance human intelligence, sense the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology of computer science that seeks to grasp the essence of intelligence and produce new smart machines that can respond in a manner similar to human intelligence. AI studies the design principles and implementation methods of various smart machines, enabling them to sense, reason, and make decisions.

[0035] Artificial intelligence technology is a comprehensive field that covers a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, pre-trained model technology, operating / interaction systems, mechatronics, and other technologies. Pre-trained models, also known as big models or foundational models, can be fine-tuned and widely applied to downstream tasks in various areas of AI. Artificial intelligence software technology primarily includes major areas such as computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0036] Computer vision is the science of how machines "see." More specifically, it uses cameras and computers to perform machine vision tasks such as recognizing, tracking, and measuring targets, and then uses graphics processing to allow computers to process images suitable for human observation or transmission to equipment for inspection. As a scientific field, computer vision studies the relevant theories and techniques for building artificial intelligence systems capable of extracting information from images or multidimensional data. Big model technology has brought about significant changes in the development of computer vision technology, and pre-trained vision models, such as swin-transformer, ViT, V-MOE, and MAE, can be rapidly and widely applied to specific downstream tasks after fine-tuning. Computer vision technology typically includes technologies such as image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / action recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, synchronous positioning and mapping, and common biometric technologies such as facial recognition and fingerprint recognition.

[0037] For example, in the present embodiment, the target model can perform image processing on the iris image to be segmented, and recognize the area where the iris is located therefrom.

[0038] Machine learning is an interdisciplinary field that encompasses multiple disciplines, including probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It focuses on how computers simulate or emulate human learning behavior, acquire new knowledge or skills, and reorganize existing knowledge structures to continually improve their performance. Machine learning is the core of artificial intelligence and the fundamental path to making computers intelligent, with applications spanning various fields of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning. Pre-trained models are the latest development in deep learning and combine the above techniques.

[0039] For example, in the embodiment of the present application, a model to be quantized can be obtained by a model training method, and accuracy evaluation and parameter adjustment for the model to be calibrated can be realized by a sample set.

[0040] The following briefly introduces application scenarios to which the technical solutions of the embodiments of the present application can be applied, and the application scenarios introduced below are for the purpose of illustrating the embodiments of the present application and are not intended to limit the scope of the present application. In the specific implementation process, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0041] The technical solution provided by the embodiments of the present application can be applied to most scenes that introduce an image segmentation model, such as image segmentation scenes in face recognition processes, and as another example, iris segmentation scenes in iris recognition processes in VR devices.

[0042] Referring to FIG. 1B, there is shown a schematic diagram of a possible application scenario provided by an embodiment of the present application, which may include a terminal device 101 and a server 102.

[0043] The terminal device 101 may be a device such as a mobile phone, a tablet computer (PAD), a personal computer (PC), an in-vehicle terminal, a device such as a camera, a video camera, a drive recorder, or a wearable device such as VR glasses or a smart watch. The server 102 may be an independent physical server, a server cluster or a distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0044] The server 102 may include one or more processors 1021, a memory 1022, an I / O interface 1023 for interacting with a terminal, etc. The server 102 may further include a database 1024, which may be used to store images to be segmented, image segmentation results, sample sets, model parameters of models to be quantized, model parameters of a target model, etc. The memory 1022 of the server 102 may further store program instructions of an image model quantization method provided by an embodiment of the present application, which, when executed by the processor 1021, can implement steps of the image model quantization method provided by an embodiment of the present application to obtain a target model and an image segmentation result, etc., and the terminal device 101 can accordingly implement the introduction of the target model or the display of the target model effect, etc.

[0045] The terminal device 101 and the server 102 may be directly or indirectly connected to each other through one or more communication networks 103. The communication network 103 may be a wired network or a wireless network, for example, a mobile cellular network or a Wireless-Fidelity (WIFI) network, or any other possible network, and the embodiments of the present application are not limited thereto.

[0046] It should be noted that the image model quantization method in the embodiments of the present application may be performed by a computer device, which may be the terminal device 101 or the server 102, i.e., the method may be performed solely by the terminal device 101 or the server 102, or jointly by the terminal device 101 and the server 102.

[0047] For example, when this method is executed solely by the server 102, the server 102 obtains the corresponding quantization target model and the computing power evaluation result of the equipment to be introduced, and then, based on the quantization target model and the computing power evaluation result, selects a target operator for the quantization target model and quantizes the target operator to obtain a target model.

[0048] In another example, when the method is executed solely by the terminal device 101, two situations may occur: the terminal device is the device to be introduced, and the terminal device is not the device to be introduced. In this case, when the method is executed by the terminal device, it can be divided into two types: directly obtaining its own computing capacity evaluation result, and obtaining the computing capacity evaluation result from another terminal device. The other execution method is similar to the execution by the server.

[0049] In another example, the method can also be completed by being jointly executed by the terminal device 101 and the server 102. In addition, the terminal device can provide a sample set, a segmentation target image, or a computing power evaluation result to the server, and the server can perform a quantization process on the target operator in the quantization target model based on the obtained computing power evaluation result and the quantization target model to obtain a corresponding target model.

[0050] Note that what is shown in FIG. 1B is merely an illustrative example, and in reality, the number of terminal devices and servers, and the communication method are not limited, and are not specifically limited in the embodiments of the present application.

[0051] In the following, combining the above application scenarios, the quantization method for an image segmentation model provided by an exemplary embodiment of the present application will be described with reference to the drawings. The above application scenarios are presented for the convenience of understanding the spirit and principles of the present application, and the embodiments of the present application are not limited in this respect.

[0052] 2 is a flowchart of an image segmentation model quantization method provided in an embodiment of the present application. The method may be performed by the aforementioned computer device. For convenience of explanation, in this embodiment, the computer device is hereinafter exemplified as a server. The specific operation steps of the method are as follows: S201: The computing power evaluation result and the quantization target model of the device to be introduced are obtained, and the computing power evaluation result is used to indicate the computing performance of the device to be introduced, and the quantization target model is used to realize image segmentation.

[0053] Specifically, when the server executes the quantization method for an image segmentation model, it must first obtain the image segmentation model to be quantized (hereinafter referred to as the quantization target model) and the calculation capacity evaluation result of the device into which the quantization target model is to be installed (the installation target device). The following different methods can be used to obtain the quantization target model.

[0054] For example, before executing the quantization method, the server can directly train an image segmentation model capable of image segmentation based on a convolutional neural network, and use the trained model as the model to be subsequently quantized. During the image segmentation model training process, the convergence conditions for the model can be set according to user needs, and this is not a limitation of the present application. After the model training process, the obtained image segmentation model itself can be directly used for image segmentation, and the accuracy and efficiency of the image segmentation can be very good. To achieve this effect, the image segmentation model must be implemented on high-cost hardware. If the image segmentation model is expected to run effectively on low-cost hardware, the server must implement the image segmentation model quantization method provided in the embodiments of the present application on the image segmentation model.

[0055] In another example, the server can directly obtain some image segmentation models that have already been trained, and if it determines that these already trained models that can directly perform image segmentation consume a similarly long amount of time to run on low-cost hardware, the server can then perform the image segmentation model quantization method provided by the embodiments of the present application on them.

[0056] After obtaining the image segmentation model through the above different methods, it is necessary to perform a quantization method on it, and all of these image segmentation models are called quantization target models.

[0057] Since the hardware parameters corresponding to different devices are different, if it is desired to ensure that the quantization target model operates well on different target devices, the server needs to perform different quantization processes on the quantization target model for the hardware parameters of different target devices, thereby achieving good technical effects on different devices. Below, we will take a target device as an example to introduce how to perform quantization processes on the quantization target model for the target device.

[0058] After completing the acquisition of the quantization target model, the server needs to evaluate the computing capabilities of the target device.

[0059] When the result of the computational capacity evaluation of a certain target device is selectively obtained, the processing method shown in FIG. 3 can be executed thereon.

[0060] Referring to FIG. 3, there is shown a flowchart of a method for obtaining a computing power evaluation result according to an embodiment of the present application. As shown in FIG. 3, the specific steps of the method are as follows: S301: A data processing test is performed on the target device based on a plurality of random data, and a test processing time length required for the target device to perform the data processing test is obtained. S302: Based on the test processing time length and the data amount corresponding to the plurality of random data, a calculation capacity evaluation result corresponding to the target device is obtained.

[0061] Specifically, when a server needs to perform a computing power test on the target device, it is necessary to focus on testing the computing power of the processor used to calculate data for the hardware device. The processor may include at least one of a central processing unit (CPU) and a graphics processing unit (GPU). The specific test target is to obtain the number of floating-point operations per second of the target device.

[0062] Based on this, when obtaining the results of the computing performance evaluation, the server can use matrix multiplication to complete the corresponding data processing test. Matrix multiplication is a common mathematical operation that can be used in fields such as image processing, signal processing, and machine learning. Since the amount of calculation required for matrix multiplication increases significantly as the matrix expands, matrix multiplication can be used to test the floating-point performance of the CPU.

[0063] For example, when a data processing test is performed on a target device, two random matrices with different sizes can be generated first, as shown in Figure 4. The sizes of these two random matrices may be preset or may be randomly generated, and the present application does not limit this. The server simply records the matrix sizes corresponding to the two random matrices.

[0064] If these two random matrices are random matrix A and random matrix B, after obtaining these two matrices, the server can obtain matrix C by performing a multiplication operation on random matrix A and random matrix B based on the hardware environment of the device to be installed.

[0065] There are two ways to perform matrix calculations based on the hardware environment of the target device. One is for the server to directly instruct the target device to perform matrix calculations, thereby obtaining the corresponding matrix C and collecting related calculation data. In this way, the server can fully collect the relevant computing performance of the target device and obtain more accurate computing performance evaluation results.

[0066] Alternatively, the server can simulate the target device according to the hardware parameters of the target device, and directly perform the relevant matrix multiplication calculation for the target device within the server to obtain relevant calculation data.In this way, the data processing test for the target device does not need to be limited by a specific target device, and the server can directly obtain the computing power evaluation result of a relatively general type of target device, and the overall evaluation result for this type of device will not be affected by the defects that exist alone in a certain target device.

[0067] The server obtains the generated matrix C, and also needs to collect the calculation time length of the matrix multiplication, so that the server can calculate the number of floating-point operations per second of the target device according to the sizes of random matrix A and random matrix B and the total test processing time length for the target device to complete the matrix calculation, and the server can use the number of floating-point operations per second as the calculation capacity evaluation result corresponding to the target device.

[0068] Optionally, when performing a data processing test on the target device to be introduced, in order to avoid any idiosyncrasies that may exist in a randomly generated matrix, the number of times the test is performed can be appropriately increased, and after performing arithmetic averaging or other statistical processing on the results of multiple tests, the obtained statistical result can be used as the computing capacity evaluation result corresponding to the target device to be introduced, thereby avoiding any adverse effects caused by accidental results.

[0069] By testing the computing power of the device to be introduced, the server obtains the computing power evaluation result of the device to be introduced, and then the server can limit the quantization direction of the model to be quantized based on the computing power evaluation result.

[0070] S202: Based on the result of the computational capacity evaluation, at least one target operator is selected from among the operators included in the model to be quantized, and the target operator has a data processing time length that reaches a set threshold in the device to be introduced.

[0071] The various operation units included in the model to be quantized are the operators described in S202, and for one model, different operators consume different amounts of calculation to complete their operation processes.

[0072] Based on this, for the GFLOPS that the target device can provide, the length of time it takes for different operators to complete data processing on the same target device may be different, and the cumulative data processing time consumed by these operators is the processing time consumed by the entire quantized target model to complete one image segmentation process on the target device.

[0073] Therefore, if the total data processing time of a model to be quantized needs to be longer than the total desired processing time set by the developer, the server needs to process the different operators included in the model to be quantized appropriately.

[0074] For example, as shown in FIG. 5, the server collects statistics on the data processing time length corresponding to when each operator completes data processing on the target device, and then selects target operators whose consumed time length exceeds a certain set threshold from among the data processing time lengths, and performs quantization processing on these target operators.

[0075] In another example, as shown in FIG. 6, the server may set a desired processing time length for each operator included in the model to be quantized. In this way, if there is an operator whose data processing time length on the target device exceeds the corresponding set threshold, the server can select target operators from these operators whose data processing time length exceeds the corresponding desired processing time length, and perform quantization processing on these target operators.

[0076] Specifically, the server can realize the selection of the target operator by implementing the selection method as shown in Figure 7. Referring to Figure 7, it is a flowchart of the target operator selection method provided in the embodiment of the present application, and as shown in Figure 7, the specific implementation steps of the method are as follows: S701: Based on the calculation capacity evaluation result, when each operator included in the model to be quantized processes data in the device to be introduced, the data processing time length corresponding to each operator is acquired.

[0077] The server can directly calculate the data processing time consumed by each operator according to the calculation capacity evaluation result of the device to be introduced (i.e., the number of floating-point operations per second) and the amount of calculation required for each operator included in the model to be quantized to complete the calculation. Alternatively, the server may perform a simulated calculation in the server according to the calculation capacity evaluation result of the device to be introduced, and directly collect the data processing time corresponding to each operator included in the model to be quantized, and this application is not limited to this.

[0078] S702: At least one target operator is selected from each operator based on the length of time for processing each data and a set threshold.

[0079] The setting thresholds may be placed individually for each operator based on the processing time length desired by the developer in advance, or the server may directly assign them to each operator in the model to be quantized based on the total desired processing time length desired by the developer; this application does not restrict this.

[0080] After obtaining the data processing time length corresponding to each operator and the corresponding set threshold, the server can select at least one target operator from each operator whose data processing time length exceeds its corresponding set threshold.

[0081] By using the above method, operators with too long data processing times can be determined as target operators from the model to be quantized. In this way, by quantizing these operators, the calculation time of the model to be quantized on the device to be introduced can be effectively reduced, and the quantization efficiency can be improved.

[0082] After completing the selection of target operators, the server can perform the following operations on these target operators: S203: Based on the difference between the data processing time length of each of the acquired at least one target operator and the corresponding desired processing time length, a quantization process is performed on the at least one target operator to acquire a corresponding quantization operator, and the data processing time length of the quantization operator in the device to be introduced is shorter than the desired processing time length.

[0083] Quantization is a technique for converting parameters of a target operator of a quantization target model (e.g., a convolution kernel, an activation function, a pooling layer, etc.) into a representation format that facilitates numerical calculation, and can improve the calculation efficiency and accuracy of the target operator in the quantization target model. Specifically, the quantization quantizes the parameters of the target operator and converts them into an 8-bit unsigned integer (e.g., 0 or 1) format for convenient subsequent calculation and processing.

[0084] The desired processing time value submitted in S203 may be equal to the threshold value set in S202, which can prevent the quantization operator obtained after the quantization process from being re-determined as the target operator in the subsequent re-screening process. Alternatively, the desired processing time value may be smaller than the threshold value set in S202, which can shorten the time required for the quantization operator obtained after the quantization process to complete data processing, thereby shortening the total time required for the entire model to complete the image segmentation process, thereby further improving the effectiveness of introducing an image segmentation model in low-cost hardware devices.

[0085] If selectively performing quantization processing on each target operator, the server can further implement the quantization processing effect thereon using the following scheme.

[0086] Specifically, the server can obtain a quantization parameter corresponding to at least one target operator based on the difference between the data processing time length of each of the obtained at least one target operator and the corresponding desired processing time length.

[0087] The quantization parameters corresponding to the target operator may include parameters such as a quantization factor or bias value, so that the quantization process performed on the target operator adjusts some of the parameters in the target operator to make the time required to complete data processing for the target operator shorter than the desired processing time.

[0088] Alternatively, the quantization process for a target operator may involve replacing the target operator with an operator that has a different structure but the same effect, so that the time length for the replaced operator to complete the operations processed by the target operator meets the desired processing time length requirement.

[0089] Then, based on these quantization parameters and the precision corresponding to at least one target operator, a quantization process is performed on the at least one target operator to obtain the corresponding quantization precision, and the error between the precision corresponding to the quantization operator and the precision corresponding to the target operator is smaller than a preset range.

[0090] When performing quantization processing on a target operator, different quantization processing methods, such as a linear quantization method, a nonlinear quantization method, or a symmetric quantization method, can be selected and used, but this application is not limited thereto, as long as the error between the accuracy of the quantization operator and the accuracy of the target operator needs to be smaller than a preset range during the quantization processing. The limit of the preset range can be determined directly by the developer or set according to parameters obtained by the server during the training process.

[0091] According to the quantization parameter determined by the required shortened time length, the quantization process for shortening the time length is appropriately performed on the target operator, while maintaining the calculation accuracy of the target operator during the quantization period, thereby achieving high-quality quantization.

[0092] S204: Transform the quantization target model into a corresponding target model based on at least one quantization operator.

[0093] After completing the acquisition of the quantization operator, the server can perform flow processing on the quantization target model according to the acquired quantization operator, thereby converting the quantization target model into a corresponding target model.

[0094] The target model achieved by the transformation may be used for deployment to the deployment target device.

[0095] In this technical proposal, a computing power test is conducted on the target equipment to be introduced, and the corresponding computing power evaluation result is obtained. Based on the computing power evaluation result, the execution status of the quantized target model on the target equipment is determined. Thus, quantization processing is appropriately performed on each operator in the quantized target model to obtain the corresponding target model, which reduces the execution time on the target equipment, improves the implementation effect on the target equipment, and reduces the difficulty of popularizing the target model.

[0096] Optionally, when converting a quantization target model into a corresponding target model based on a quantization operator, the server can specifically perform the following operations to further reduce the processing time required when the target model performs image segmentation on the target device:

[0097] Referring to FIG. 8, it is a flowchart of a method for adjusting model accuracy provided in an embodiment of the present application. As shown in FIG. 8, the specific implementation steps of the method are as follows: S801: Construct a corresponding intermediate model based on at least one quantization operator and other operators in a model to be quantized, excluding at least one target operator.

[0098] After the server completes the quantization process for the target operators, the server can jointly construct an intermediate model using these acquired quantization operators and the quantization operators originally included in the quantization target model whose data processing time length does not exceed the corresponding set threshold.

[0099] To further reduce the time it takes for the target model to process the image segmentation process on the target device, the server can further reduce the model precision corresponding to the model (e.g., reduce the precision of the data processed by the model), thereby reducing the number of calculations the model performs when performing image segmentation, and thereby reducing the length of time it takes for the model to process data.

[0100] S802: Determine a target accuracy corresponding to the target model based on the calculation capacity evaluation result and the total desired processing time length corresponding to the target model.

[0101] S803: Adjust the model accuracy of the intermediate model based on the target accuracy to obtain the corresponding target model.

[0102] Specifically, as shown in FIG. 9, the server must first determine the model accuracy that the desired target model will achieve, and this determination process can be determined based on the computing power evaluation results corresponding to the equipment to be introduced and the total desired processing time length corresponding to the target model.

[0103] For example, it is determined whether the computing performance of the target device is insufficient to ensure that the original floating point (FP) 32 precision must be converted to integer (INT) 8 precision when the total data processing time of the target model is to be achieved within the corresponding total desired processing time. In another example, when it is determined that the computing power of the target device is sufficient to complete high-precision data processing within the total desired processing time, the server can convert the original FP32 precision model precision to FP16 precision. In this way, even when a certain calculation precision is reserved, the computing performance of the target device can be fully utilized and the possibility of unused performance can be reduced.

[0104] Optionally, after completing the accuracy adjustment for the intermediate model, the server can further perform model training once based on the calibrated target model after accuracy adjustment, thereby achieving both accuracy and time consumption when the target model performs image segmentation on the target device.

[0105] Referring to FIG. 10A, this is a flowchart of a method for training a calibration target model provided by an embodiment of the present application. As shown in FIG. 10A, the specific implementation steps of the method are as follows: S1001: A sample set is obtained, and each sample in the sample set includes a sample segmentation image and a corresponding sample segmentation result.

[0106] The sample segmentation result corresponding to the sample segmented image of the sample set may be the image segmentation result obtained after the quantization target model performs image segmentation processing. In this way, when performing subsequent training, the sample segmentation result is used as a tag for training, which can maximize the assurance that the image segmentation result obtained by the calibration target model after training can be maintained to correspond to the segmentation result between the initial quantization target model.

[0107] S1002: The calibration target model is set in an execution environment with the same hardware parameters as the device to be introduced, and the test segmentation result after the calibration target model performs image segmentation processing on the sample segmented image and the image segmentation time length for which the calibration target model completes the image segmentation processing are obtained.

[0108] Before training the calibration target model, the calibration target model must first be set in an execution environment that is the same as the hardware parameters of the device to be installed. To ensure the convergence efficiency of model training, the server builds a simulation environment by itself, so that the environment is the same as the execution environment of the device to be installed. Then, the calibration target model is trained in the simulation environment, so that the calibration target model performs a segmentation process on the sample segmented image and obtains the corresponding test segmentation result, and at the same time, the image segmentation time length corresponding to the calibration target model completing the image segmentation process can be obtained.

[0109] S1003: Based on the error between the test segmentation result and the sample segmentation result, and the error between the image segmentation time length and the total desired processing time length, perform model parameter adjustment on the calibration target model to obtain the corresponding target model.

[0110] As shown in Figure 10B, during one round of training, the server can adjust model parameters for the model to be calibrated according to the loss value between the test segmentation result corresponding to a certain acquired sample segmented image and the sample segmentation result, and the error between the image segmentation time length consumed by the model to be calibrated when processing the sample segmented image and the total desired processing time length corresponding to the model to be calibrated, thereby completing one round of training for the model to be calibrated.

[0111] In this way, after undergoing multiple rounds of training as described above, the server can obtain a target model in which the entire model can simultaneously achieve both accuracy and efficiency in image segmentation.

[0112] The above describes the process of acquiring a target model. After the server completes the acquisition of the target model, the server can use the target model to complete the installation and image processing in the device to which it is installed. For example, the server can install the target model in a VR device, such as for the iris segmentation process during iris recognition, or the server can install it in a low-cost mobile phone to realize the face image segmentation process during face recognition.

[0113] Referring to FIG. 11, there is shown a flowchart of an image segmentation method provided in an embodiment of the present application, and the specific implementation steps of the method are as follows: S1101: The target model is set to an execution environment with the same hardware parameters as the device to be introduced.

[0114] In addition, when introducing a target model, the object to be introduced is not limited to one device, namely, the object to be introduced, but may be introduced into other devices with the same hardware parameters as the object to be introduced, or may be introduced directly into a virtual machine; this application does not restrict this.

[0115] S1102: After the server completes the introduction to the target model, it can perform image segmentation processing on the iris image to be segmented based on at least one quantization operator in the target model and other operators other than at least one target operator in the quantization target model, and obtain the corresponding iris image segmentation result.

[0116] As shown in Figure 12, after completing the introduction into the target model, when performing segmentation processing on an iris image, the target model can obtain the corresponding iris image segmentation result by performing image segmentation processing on the iris image to be segmented based on the quantization operator after quantization processing contained in the target model itself and other operators other than the target operator that do not require quantization processing.

[0117] In this way, the processing time required to perform iris image segmentation on the device to be introduced is reduced, ensuring the effectiveness of iris recognition on low-cost devices, thereby making it possible to popularize the application of iris recognition technology on low-cost hardware devices.

[0118] In order to further reduce the processing time for iris image segmentation, the accuracy of the target model can be set to a preset accuracy, in which case the preset accuracy may be the INT8 accuracy or FP16 accuracy as described above, thereby further reducing the processing time required when a target model of that accuracy performs image segmentation.

[0119] To match such model accuracy, adaptive accuracy adjustments can be performed in advance on the images input to the target model, thereby ensuring a match between the data input to the model and the accuracy of the model.

[0120] Please refer to FIG. 13A, which is a flowchart of an image segmentation method provided in an embodiment of the present application. As shown in FIG. 13A, the specific implementation steps of the method are as follows: S1301: Based on a preset accuracy, the image accuracy corresponding to the iris image to be segmented is adjusted, and a corresponding intermediate image is obtained, and the image accuracy of the intermediate image corresponds to the preset accuracy.

[0121] The preset precision is the model precision corresponding to the target model. If the preset precision is INT8, the server needs to adjust the image precision of the iris image to be segmented from the possible FP32 precision or FP16 precision to the corresponding INT8 precision, so that it can be input into the target model for image segmentation processing.

[0122] S1302: Perform image segmentation processing on the intermediate image based on at least one quantization operator in the target model and other operators in the quantization target model excluding the at least one target operator, to obtain a corresponding intermediate segmentation result.

[0123] In this case, the target model with a predetermined accuracy performs image segmentation processing on the iris image to be segmented with the predetermined accuracy, and the accuracy of the intermediate segmentation result obtained is also the predetermined accuracy. However, since the image accuracy of the intermediate segmentation result with the predetermined accuracy does not match, it cannot be directly used as the image segmentation result corresponding to the original iris image to be segmented with high accuracy. Therefore, the server needs to perform the following operations to output the final iris image segmentation result:

[0124] S1303: Based on the image accuracy corresponding to the iris image to be segmented, accuracy adjustment is performed on the intermediate segmentation result to obtain a corresponding iris image segmentation result, and the image accuracy corresponding to the iris image segmentation result is the same as the image accuracy corresponding to the iris image to be segmented.

[0125] After the target model has completed the low-precision segmentation process for the image to be segmented, the server can further perform inverse quantization on the obtained low-precision intermediate segmentation result, for example, inverse quantization of the low-precision INT8 precision result into a result with FP32 precision or FP16 precision, which can improve the accuracy and readability of the iris image segmentation result and meet the accuracy requirements of the subsequent image processing steps, thereby improving the feasibility of the technical solution.

[0126] As shown in Figure 13B, the iris image to be segmented, which has an image precision of FP32, is first subjected to precision conversion to obtain an intermediate image with an image precision of INT8, and then the intermediate image is input into the target model to obtain a corresponding intermediate segmentation result with a precision of INT8. Finally, the intermediate segmentation result with INT8 precision can be converted into an iris image segmentation result with a precision of FP32.

[0127] The above describes the quantization process for an image model and some possible embodiments thereof provided by the embodiments of the present application. The quantization process is introduced below to clarify the technical solutions obtained after possible combinations of the above methods.

[0128] Referring to FIG. 14, it is a flowchart of an image segmentation model quantization method provided in an embodiment of the present application, and the specific implementation steps of the method are as follows: S1401: A quantization target model is acquired, and an introduction target device that requires the quantization target model to be introduced is determined. S1402: A data processing test is performed on the target device based on a plurality of random data, and a test processing time length required for the target device to complete the data processing test is obtained. S1403: The result of the evaluation of the computing capacity of the target device is determined based on the test processing time length and the amount of random data. S1404: Based on the result of the calculation capacity evaluation, at least one target operator included in the model to be quantized is selected, and the data processing time length of the target operator in the device to be introduced reaches a set threshold. S1405: Based on the difference between the data processing time length of each of the acquired at least one target operator and the corresponding desired processing time length, a quantization process is performed on the at least one target operator to acquire a corresponding quantization operator, and the data processing time length of the quantization operator in the target device to be introduced is shorter than the desired processing time length. S1406: Construct a corresponding intermediate model based on at least one quantization operator and other operators in the model to be quantized, excluding at least one target operator. S1407: Determine a target accuracy corresponding to the target model based on the calculation capacity evaluation result and the total desired processing time length corresponding to the target model. S1408: Based on the target accuracy, adjust the model accuracy of the intermediate model to obtain the corresponding calibration target model. S1409: A sample set is obtained, and each sample in the sample set includes a sample segmentation image and a corresponding sample segmentation result. S1410: The calibration target model is set in an execution environment with the same hardware parameters as the device to be introduced, and the test segmentation result after the calibration target model performs image segmentation processing on the sample segmented image and the image segmentation time length for which the calibration target model completes the image segmentation processing are obtained. S1411: Based on the error between the test segmentation result and the sample segmentation result, and the error between the image segmentation time length and the total desired processing time length, perform model parameter adjustment on the calibration target model to obtain the corresponding target model.

[0129] As shown in FIG. 15, after the target model is obtained, the server can set the target model in the target device so as to complete the image segmentation process of the corresponding iris image.

[0130] Based on the same inventive concept, an embodiment of the present application further provides an image segmentation model quantization device. Refer to FIG. 16, which is a structural schematic diagram of an image segmentation model quantization device provided by an embodiment of the present application. The device may be the above-mentioned terminal device or server, or a chip or integrated circuit therein, etc., and the device includes a module / unit / technical means for performing the method performed by the terminal device or server in the above-mentioned method embodiment.

[0131] Illustratively, the device 1600 includes: an acquisition module 1601 for acquiring a computing power evaluation result and a quantization target model of the target device, the computing power evaluation result being used to indicate the computing performance of the target device, and the quantization target model being used to realize image segmentation; A selection module 1602 for selecting at least one target operator from each operator included in the quantization target model based on the calculation performance evaluation result, wherein the target operator is selected by the selection module 1602 so that the data processing time length in the target device reaches a set threshold; and a quantization module 1603 for performing a quantization process on at least one target operator based on a difference between the data processing time length of each of the acquired at least one target operator and a corresponding desired processing time length to obtain a corresponding quantization operator, wherein the quantization operator has a data processing time length in an installation target device that is shorter than the desired processing time length, and converting a quantization target model into a corresponding target model based on the at least one quantization operator, the target model being used for installation in the installation target device.

[0132] In a possible embodiment, the quantization module 1603, when used to transform a quantization target model based on at least one quantization operator, specifically: constructing a corresponding intermediate model based on the at least one quantization operator and other operators in the model to be quantized, excluding the at least one target operator; determining a target accuracy corresponding to the target model based on the calculation performance evaluation result and the total desired processing time length corresponding to the target model; Based on the target accuracy, the model accuracy of the intermediate model is adjusted and used to obtain the corresponding target model.

[0133] In a possible embodiment, the quantization module 1603 adjusts the model precision of the intermediate model based on the target precision, and when used to obtain the corresponding target model, specifically: According to the target accuracy, adjust the model accuracy of the intermediate model and obtain the corresponding calibration target model; A sample set is obtained, and each sample in the sample set includes a sample segmentation image and a corresponding sample segmentation result; The calibration target model is set in an execution environment with the same hardware parameters as the target device, and the test segmentation result after the calibration target model performs image segmentation processing on the sample segmented image and the image segmentation time length for which the calibration target model completes the image segmentation processing are obtained. Based on the error between the test segmentation result and the sample segmentation result, and the error between the image segmentation time length and the total desired processing time length, model parameter adjustment is performed on the calibration target model to obtain the corresponding target model.

[0134] In a possible embodiment, when the acquisition module 1601 is used to acquire the calculation capacity evaluation result of the target device, the acquisition module 1601 specifically: A data processing test is performed on the target device based on the plurality of random data, and a test processing time length required for the target device to perform the data processing test is obtained; It is used to obtain a computing capacity evaluation result corresponding to the target equipment based on the test processing time length and the data amount corresponding to the plurality of random data.

[0135] In a possible embodiment, when the selection module 1602 is used to select at least one target operator from the operators included in the model to be quantized based on the calculation performance evaluation result, the selection module 1602 specifically selects: Based on the calculation capacity evaluation result, a data processing time length corresponding to each operator included in the quantization target model when the operator processes data on the target device is obtained; It is used to select at least one target operator from each operator based on the data processing time length and a set threshold.

[0136] In a possible embodiment, the quantization module 1603 performs a quantization process on at least one target operator based on a difference between the data processing time length of each of the obtained at least one target operator and the corresponding desired processing time length, and when used to obtain a corresponding quantization operator, specifically: Obtaining a quantization parameter corresponding to the at least one target operator based on a difference between the obtained data processing time length of each of the at least one target operator and the corresponding desired processing time length; Based on the quantization parameter and the precision corresponding to the at least one target operator, a quantization process is performed on the at least one target operator to obtain a corresponding quantization operator, and an error between the precision corresponding to the quantization operator and the precision corresponding to the target operator is smaller than a preset range.

[0137] In a possible embodiment, the apparatus 1600 further includes a processing module 1604, where after obtaining the target model, the processing module 1604: The target model is set to the same execution environment as the hardware parameters of the device to be introduced. It is used to perform image segmentation processing on the iris image to be segmented based on at least one quantization operator in the target model and other operators in the quantization target model excluding the at least one target operator, and obtain a corresponding iris image segmentation result.

[0138] In a possible embodiment, the model accuracy corresponding to the target model is a preset accuracy; When the processing module 1604 is used to perform image segmentation processing on the segmentation target iris image based on at least one quantization operator in the target model and other operators other than the at least one target operator in the quantization target model, and obtain a corresponding iris image segmentation result, specifically: Adjusting the image precision corresponding to the iris image to be segmented based on the preset precision, and obtaining a corresponding intermediate image, wherein the image precision of the intermediate image corresponds to the preset precision; Performing an image segmentation process on the intermediate image based on at least one quantization operator in the target model and other operators in the quantization target model except for the at least one target operator, to obtain a corresponding intermediate segmentation result; Based on the image accuracy corresponding to the iris image to be segmented, accuracy adjustment is performed on the intermediate segmentation result to obtain a corresponding iris image segmentation result, and the image accuracy corresponding to the iris image segmentation result is the same as the image accuracy corresponding to the iris image to be segmented.

[0139] As an embodiment, the device described in FIG. 16 can be used to execute the method described in the embodiment shown in FIG. 2. Therefore, for the functions that can be realized by each functional module of the device, the description of the embodiment shown in FIG. 2 can be referred to, and will not be described in detail here.

[0140] The memory 1701 is used to store computer programs executed by the processor 1702. The memory 1701 may mainly include a program storage area and a data storage area, the program storage area may store an operating system and programs necessary for executing instant messaging functions, and the data storage area may store various types of instant messaging information and operating instruction sets.

[0141] Memory 1701 may be volatile memory, such as random-access memory (RAM), non-volatile memory, such as read-only memory, flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), or any other medium that can be used to carry or store a desired computer program in the form of instructions or data structures and that can be accessed by a computer, but is not limited to this. Memory 1701 may also be a combination of the above memories.

[0142] The processor 1702 may include one or more central processing units (CPUs), or may be a digital processing unit, etc. The processor 1702 implements the quantization method of the image segmentation model when calling a computer program stored in the memory 1701.

[0143] The communication module 1703 is used to communicate with terminal devices and other servers.

[0144] In the embodiment of the present application, the specific connection medium between the memory 1701, communication module 1703, and processor 1702 is not limited. In the embodiment of the present application, the memory 1701 and the processor 1702 are connected via a bus 1704, which is shown by a bold line in FIG. 17. The connection method between other components is merely for schematic illustration and is not intended to be limiting. The bus 1704 can be divided into an address bus, a data bus, a control bus, etc. For convenience of description, only one bold line is shown in FIG. 17, but this does not indicate that there is only one bus or only one type of bus.

[0145] The memory 1701 stores a computer storage medium, and the computer storage medium stores computer-executable instructions, which are used to implement the quantization method of the image segmentation model of the embodiment of the present application. The processor 1702 is used to execute the quantization method of the image segmentation model.

[0146] In another embodiment, the electronic device may be another electronic device, such as the terminal device 101 shown in FIG. 1B. In this embodiment, the structure of the electronic device may include components such as a communication component 1810, a memory 1818, a display unit 1830, a camera 1840, a sensor 1850, an audio circuit 1860, a Bluetooth module 1870, and a processor 1880, as shown in FIG.

[0147] The communication component 1810 is used to communicate with the server. In some embodiments, the circuit may include a Wireless Fidelity (WiFi) module, which is a short-range wireless transmission technology, and the electronic device helps the object to send and receive information through the WiFi module.

[0148] The memory 1818 may be used to store software programs and data. The processor 1880 executes the software programs or data stored in the memory 1818 to perform various functions and data processing of the terminal device 101.

[0149] The display unit 1830 may further be used to display information entered by or provided to the objects, as well as a graphical user interface (GUI) of various menus of the terminal device 101.

[0150] The physical terminal device may further include at least one type of sensor 1850, such as an acceleration sensor 1851, a distance sensor 1852, a fingerprint sensor 1853, or a temperature sensor 1854. The terminal device may further include other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, a light sensor, or a motion sensor.

[0151] The audio circuit 1860 , the speaker 1861 , and the microphone 1862 can provide an audio interface between the object and the terminal device 101 .

[0152] The processor 1880 is the control center of the physical terminal device. It connects various parts of the entire terminal using various interfaces and lines, and executes various functions and processes data of the terminal device by running or executing software programs stored in the memory 1818 and accessing data stored in the memory 1818. In some embodiments, the processor 1880 may include one or more processing units. The processor 1880 may further integrate an application processor and a baseband processor. The application processor mainly processes the operating system, user interface, and application programs, and the baseband processor mainly processes wireless communication. Note that the baseband processor does not have to be integrated into the processor 1880. The processor 1880 in this application can execute the operating system, application programs, user interface display and touch response, and methods of the embodiments of this application. The processor 1880 is also coupled to the display unit 1830.

[0153] It should be noted that in specific embodiments of the present application, with respect to object data related to image segmentation models, etc., when the above examples of the present application are applied to specific products or technologies, object permission or consent must be obtained, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0154] Moreover, the embodiments of the present application further provide a storage medium, which is used to store a computer program, and which is used to execute the method provided by the above embodiments.

[0155] An embodiment of the present application further provides a computer program product including a computer program, which, when run on a computer, causes the computer to perform the method provided by the above embodiment.

[0156] It should be noted that although the above detailed description describes several units or subunits of the device, such division is not required and is merely exemplary. In fact, depending on the embodiment of the present application, the features and functions of two or more units described above may be embodied in one unit. Conversely, the features and functions of one unit described above may be further divided to be embodied by multiple units.

[0157] Furthermore, although the operations of the methods herein are described in a particular order in the figures, this does not require or imply that the operations must be performed in that particular order, or that all of the illustrated operations must be performed to achieve desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into a single step, and / or a single step may be broken down into multiple steps.

[0158] Those skilled in the art will appreciate that the present application may be provided as a method, a system, or a computer program product. Therefore, the present application may utilize an entirely hardware embodiment, an entirely software embodiment, or an embodiment that combines software and hardware. Furthermore, the present application may utilize a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk memory, CD-ROM, optical memory, etc.) that contain computer-usable program code.

[0159] The present application is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to the present application. Each process and / or block of the flowcharts and / or block diagrams, and combinations of processes and / or blocks in the flowcharts and / or block diagrams, may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, whereby the instructions executed by the processor of the computer or other programmable data processing device produce an apparatus for implementing the function specified in the process or processes of the flowcharts and / or the block or blocks of the block diagrams.

[0160] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture that includes an instruction apparatus that implements the functions specified in a process or processes of the flowcharts and / or a block or blocks of the block diagrams.

[0161] These computer program instructions may be loaded into a computer or other programmable data processing device, which causes the computer or other programmable device to perform a series of operational steps to produce a computer-implemented process, whereby the instructions executing on the computer or other programmable device provide steps for implementing the functions specified in the process or processes of the flowcharts and / or the block or blocks of the block diagrams.

[0162] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application also intends to include these modifications and variations. [Explanation of symbols]

[0163] 101 Terminal Equipment 102 Server 103 Communication Network 1021 processor 1022 memory 1023 I / O interfaces 1024 databases 1600 equipment 1601 Acquisition Module 1602 Sorting Module 1603 Quantization Module 1604 Processing Module 1701 memory 1702 processor 1703 Communication Module 1704 Bus 1810 Communication Components 1818 memory 1830 display unit 1840 Camera 1850 Sensor 1851 Acceleration Sensor 1852 distance sensor 1853 Fingerprint Sensor 1854 temperature sensor 1860 Audio Circuit 1861 Speaker 1862 microphone 1870 Bluetooth Module 1880 processor

Claims

1. 1. A method for quantizing an image segmentation model executed by a computing device, the method comprising: a step of acquiring a computing power evaluation result and a quantization target model of the introduction target device, the computing power evaluation result being used to indicate the computing performance of the introduction target device, and the quantization target model being used to realize image segmentation; a step of selecting at least one target operator from among the operators included in the model to be quantized based on the calculation performance evaluation result, wherein the target operator is selected when a data processing time length in the device to be introduced reaches a set threshold; a step of performing a quantization process on the at least one target operator based on a difference between a data processing time length of each of the acquired at least one target operator and a corresponding desired processing time length, and acquiring a corresponding quantization operator, wherein the data processing time length of the quantization operator in the target device is shorter than the desired processing time length; converting the quantized target model into a corresponding target model based on the at least one quantization operator, the target model being used for deployment to the deployment target device; A method characterized by:

2. The step of transforming the quantization target model based on the at least one quantization operator includes: constructing a corresponding intermediate model based on the at least one quantization operator and other operators in the model to be quantized except for the at least one target operator; determining a target accuracy corresponding to the target model based on the calculation performance evaluation result and a total desired processing time length corresponding to the target model; adjusting model accuracy of the intermediate model based on the target accuracy to obtain a corresponding target model; 2. The method of claim 1 .

3. The step of adjusting the model accuracy of the intermediate model based on the target accuracy and obtaining a corresponding target model includes: adjusting the model accuracy of the intermediate model based on the target accuracy to obtain a corresponding calibration target model; obtaining a sample set, each sample in the sample set including a sample segmentation image and a corresponding sample segmentation result; A step of setting the calibration target model in an execution environment with the same hardware parameters as the device to be introduced, and acquiring a test segmentation result after the calibration target model performs image segmentation processing on the sample segmented image, and an image segmentation time length for which the calibration target model completes the image segmentation processing; and performing model parameter adjustment on the calibration target model based on an error between the test segmentation result and the sample segmentation result and an error between the image segmentation time length and the total desired processing time length to obtain a corresponding target model.

3. The method of claim 2.

4. The step of acquiring the calculation capacity evaluation result of the device to be introduced includes: A step of performing a data processing test on the introduction target device based on a plurality of random data, and acquiring a test processing time length required for the introduction target device to perform the data processing test; and acquiring a computing capacity evaluation result corresponding to the target device based on the test processing time length and the amount of data corresponding to the plurality of random data. The method according to any one of claims 1 to 3.

5. The step of selecting at least one target operator from among the operators included in the model to be quantized based on the calculation performance evaluation result includes: A step of acquiring a data processing time length corresponding to each operator included in the quantization target model when the operator processes data in the introduction target device based on the calculation capacity evaluation result; and selecting the at least one target operator from the operators based on the data processing time length and the set threshold.

5. The method of claim 4.

6. The step of performing a quantization process on the at least one target operator based on a difference between the data processing time length of each of the acquired at least one target operator and the corresponding desired processing time length to acquire a corresponding quantization operator includes: obtaining a quantization parameter corresponding to the at least one target operator based on a difference between the obtained data processing time length of each of the at least one target operator and a corresponding desired processing time length; performing a quantization process on the at least one target operator based on the quantization parameter and a precision corresponding to the at least one target operator to obtain a corresponding quantization operator, wherein an error between the precision corresponding to the quantization operator and the precision corresponding to the target operator is smaller than a preset range; 2. The method of claim 1 .

7. After obtaining the target model, the method comprises: setting the target model in an execution environment that has the same hardware parameters as the target device; performing an image segmentation process on the iris image to be segmented based on at least one quantization operator in the target model and other operators in a quantization target model excluding the at least one target operator, and obtaining a corresponding iris image segmentation result; 7. The method according to claim 1, wherein the first and second electrodes are connected to a first electrode.

8. the model accuracy corresponding to the target model is a preset accuracy; The step of performing an iris image segmentation process on an image to be segmented based on at least one quantization operator in the target model and other operators in a model to be quantized other than the at least one target operator, and obtaining a corresponding iris image segmentation result, a step of adjusting an image precision corresponding to the iris image to be segmented based on the predetermined precision, and acquiring a corresponding intermediate image, wherein the image precision of the intermediate image corresponds to the predetermined precision; performing an image segmentation process on the intermediate image based on at least one quantization operator in the target model and other operators in a model to be quantized other than the at least one target operator, to obtain a corresponding intermediate segmentation result; and performing precision adjustment on the intermediate segmentation result based on the image precision corresponding to the segmentation target image to obtain a corresponding iris image segmentation result, wherein the image precision corresponding to the iris image segmentation result is the same as the image precision corresponding to the segmentation target iris image.

8. The method of claim 7.

9. 1. An apparatus for quantizing an image segmentation model, the apparatus comprising: an acquisition module for acquiring a computing power evaluation result and a quantization target model of a target device for introduction, the computing power evaluation result being used to indicate the computing performance of the target device for introduction, and the quantization target model being used to realize image segmentation; a selection module for selecting at least one target operator from among the operators included in the model to be quantized based on the calculation performance evaluation result, wherein the target operator is a selection module for which the data processing time length in the device to be introduced reaches a set threshold; a quantization module for performing a quantization process on the at least one target operator based on a difference between a data processing time length of each of the acquired at least one target operator and a corresponding desired processing time length to obtain a corresponding quantization operator, wherein the data processing time length in the introduction target device is shorter than the desired processing time length, and the quantization module converts the quantization target model into a corresponding target model based on the at least one quantization operator, and the target model is used for introduction into the introduction target device; An apparatus characterized in that

10. An electronic device comprising a processor and a memory, the memory storing program code which, when executed by the processor, causes the processor to perform the steps of the method of any one of claims 1 to 8. An electronic device characterized by:

11. A computer-readable storage medium used to store a computer program, which, when executed on a computing device, causes the computing device to perform the steps of the method according to any one of claims 1 to 8. A computer-readable storage medium comprising:

12. A computer program product comprising a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 8.

1. A computer program product comprising:

Images