A model generation method and device, an image processing method and device, and an electronic device

Abstract

Embodiments of the present application provide a model generation method and device, an image processing method and device, and an electronic device. The scheme is as follows: obtaining a first image; selecting a plurality of resolutions from a preset resolution range of an image processing device as first resolutions; for each first resolution, performing inference on a first deep learning model corresponding to the first resolution according to the first image to obtain an inference result of the first deep learning model corresponding to the first resolution; determining, according to the accuracy of the inference result of each first deep learning model corresponding to the first resolution, a first resolution corresponding to a preset number of first deep learning models with the highest accuracy as an optimal resolution of the image processing device; and generating a second deep learning model of the image processing device according to the first deep learning model corresponding to each optimal resolution. Through the technical scheme provided by the embodiments of the present application, the resolution supported by the deep learning model is increased, and the accuracy of the image processing result is improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a model generation, image processing method, apparatus, and electronic device. Background Technology

[0002] In the process of image processing, since the resolution of images acquired by different imaging devices is different, the image processing device needs to support the processing of images with different resolutions when processing the images acquired by the imaging device.

[0003] In related technologies, when image processing devices process images, due to limited hardware resources, they often only utilize a single, relatively high-resolution deep learning model to process the image and obtain the processing result. When using this deep learning model for image processing, if the error between the resolution of the image to be processed and the resolution supported by the deep learning model is large, the accuracy of the image processing result determined by the deep learning model will be greatly reduced. Summary of the Invention

[0004] The purpose of this application is to provide a model generation, image processing method, apparatus, and electronic device to increase the resolution supported by deep learning models and improve the accuracy of image processing results. The specific technical solution is as follows:

[0005] This application provides a model generation method applied on a server side, the method comprising:

[0006] Get the first image;

[0007] Multiple resolutions are selected from the preset resolution range of the image processing device as the first resolution;

[0008] For each first resolution, inference is performed on the first deep learning model corresponding to the first resolution based on the first image to obtain the inference result of the first deep learning model corresponding to the first resolution;

[0009] Based on the accuracy of the inference results of the first deep learning model corresponding to each first resolution, the first resolution corresponding to the preset number of first deep learning models with the highest accuracy is determined as the optimal resolution of the image processing device.

[0010] A second deep learning model for the image processing device is generated based on the first deep learning model corresponding to each optimal resolution.

[0011] Optionally, the step of selecting multiple resolutions from a preset resolution range of the image processing device as the first resolution includes:

[0012] The resolution selection interval is determined based on the image alignment method supported by the image processing device.

[0013] According to the resolution selection interval, multiple resolutions are selected from the preset resolution range of the image processing device as the first resolution.

[0014] Optionally, if the first deep learning model is an object detection model, then the step of performing inference on the first deep learning model corresponding to each first resolution based on the first image to obtain the inference result of the first deep learning model corresponding to the first resolution includes:

[0015] For each first resolution, the target object in the first image is detected using the first deep learning model corresponding to that first resolution, and the detection result of the first deep learning model corresponding to that first resolution is obtained.

[0016] The step of determining the first resolution corresponding to a preset number of first deep learning models with the highest accuracy as the optimal resolution of the image processing device based on the accuracy of the inference results of the first deep learning model corresponding to each first resolution includes:

[0017] Based on the detection results of the first deep learning model corresponding to each first resolution, determine the detection rate and / or intersection-over-union (IOU) index of the first deep learning model for the target object in the first image for the first resolution;

[0018] The first resolution corresponding to the highest preset number of first deep learning models with the highest detection rate and / or IOU index is determined as the optimal resolution of the image processing device.

[0019] Optionally, the step of detecting objects in the first image using a first deep learning model corresponding to each first resolution, and obtaining the detection result of the first deep learning model corresponding to the first resolution, includes:

[0020] For each first resolution, if the first resolution is greater than the second resolution of the first image, then pixels with preset values ​​are added to the right side of the right boundary and the bottom side of the bottom boundary of the first image to obtain a second image with the resolution of the first resolution.

[0021] Using the first deep learning model corresponding to the first resolution, the target objects in the second image are detected, and the detection results of the first deep learning model corresponding to the first resolution are obtained.

[0022] Optionally, the step of generating a second deep learning model for the image processing device based on a first deep learning model corresponding to each optimal resolution includes:

[0023] For each optimal resolution, the first deep learning model corresponding to the optimal resolution is compiled according to the model format that the image processing device can recognize, to obtain the third deep learning model of the image processing device.

[0024] By packaging the third deep learning models, we obtain the second deep learning model.

[0025] This application also provides an image processing method applied to an image processing device, the method comprising:

[0026] The second deep learning model of the image processing device is obtained from the server. The second deep learning model is generated based on the first deep learning model corresponding to a preset number of optimal resolutions of the image processing device. The preset number of optimal resolutions are determined based on the accuracy of the inference results of the first deep learning models corresponding to multiple first resolutions in the preset resolution range of the image processing device.

[0027] Obtain the image to be processed;

[0028] Based on the third resolution of the image to be processed, the image is processed using the second deep learning model to obtain the image processing result.

[0029] Optionally, the second deep learning model includes the preset number of third deep learning models corresponding to the optimal resolution, and each third deep learning model corresponding to the optimal resolution is compiled from the first deep learning model corresponding to the optimal resolution.

[0030] The step of processing the image to be processed based on the third resolution of the image to be processed and obtaining the image processing result according to the second deep learning model includes:

[0031] Based on the optimal resolution corresponding to each of the third deep learning models in the second deep learning model, the optimal resolution with the smallest error between the resolution and the third resolution of the image to be processed is determined as the target resolution, and the target resolution is greater than or equal to the third resolution;

[0032] If the target resolution is greater than the third resolution, then pixels with preset values ​​are added to the right side of the right boundary and the bottom side of the bottom boundary of the image to be processed to obtain a third image with the target resolution.

[0033] The third image is processed using the third deep learning model corresponding to the target resolution in the second deep learning model to obtain the image processing result.

[0034] This application embodiment also provides a model generation apparatus, applied on a server side, the apparatus comprising:

[0035] The first acquisition module is used to acquire the first image;

[0036] The selection module is used to select multiple resolutions from the preset resolution range of the image processing device as the first resolution;

[0037] The inference module is used to perform inference on the first deep learning model corresponding to each first resolution based on the first image, and obtain the inference result of the first deep learning model corresponding to the first resolution.

[0038] The determining module is used to determine the first resolution corresponding to the preset number of first deep learning models with the highest accuracy as the optimal resolution of the image processing device based on the accuracy of the inference results of the first deep learning model corresponding to each first resolution.

[0039] The generation module is used to generate a second deep learning model for the image processing device based on the first deep learning model corresponding to each optimal resolution.

[0040] This application also provides an image processing apparatus, applied to an image processing device, the apparatus comprising:

[0041] The second acquisition module is used to acquire the second deep learning model of the image processing device from the server. The second deep learning model is generated based on the first deep learning model corresponding to a preset number of optimal resolutions of the image processing device. The preset number of optimal resolutions are determined based on the accuracy of the inference results of the first deep learning models corresponding to multiple first resolutions in the preset resolution range of the image processing device.

[0042] The third acquisition module is used to acquire the image to be processed;

[0043] The processing module is used to process the image to be processed based on the second deep learning model according to the third resolution of the image to be processed, so as to obtain the image processing result.

[0044] This application also provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0045] Memory, used to store computer programs;

[0046] When a processor executes a program stored in memory, it implements any of the steps of the model generation method described above.

[0047] This application also provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0048] Memory, used to store computer programs;

[0049] When executing a program stored in memory, the processor implements any of the image processing method steps described above.

[0050] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the model generation method steps described above.

[0051] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the image processing method steps described above.

[0052] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to execute any of the model generation methods described above.

[0053] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to perform any of the image processing methods described above.

[0054] Beneficial effects of the embodiments in this application:

[0055] The technical solution provided in this application, in the model generation stage, involves the server selecting multiple first resolutions from a preset resolution range of the image processing device. Based on the accuracy of the inference results of the first deep learning model corresponding to each first resolution, the server determines the first resolution corresponding to a preset number of first deep learning models with the highest accuracy as the optimal resolution for the image processing device. Then, based on the first deep learning model corresponding to each optimal resolution, a second deep learning model for the image processing device is generated. In the image processing stage, the image processing device processes the image to be processed based on the third resolution of the image to be processed and the second deep learning model generated in the model generation stage to obtain the image processing result.

[0056] Compared to related technologies, since the second deep learning model generated during the model generation stage is generated based on the first deep learning model corresponding to a preset number of optimal resolutions, the second deep learning model generated during the model generation stage can support a preset number of optimal resolutions. This allows the second deep learning model to support multiple resolutions, increasing the resolutions supported by the deep learning model.

[0057] Furthermore, in the image processing stage, since the second deep learning model used by the image processing device can support a preset number of optimal resolutions instead of just a fixed resolution, there must be an optimal resolution among the resolutions supported by the second deep learning model that has the smallest error with the resolution of the image to be processed. At this time, using the second deep learning model to process the image to be processed will effectively reduce the error between the resolution supported by the deep learning model and the resolution of the image to be processed, and improve the accuracy of the image processing results.

[0058] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0059] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0060] Figure 1 This is a schematic diagram of a first flowchart of the model generation method provided in an embodiment of this application;

[0061] Figure 2 This is a second flowchart illustrating the model generation method provided in the embodiments of this application;

[0062] Figure 3 This is a schematic diagram of a third process for the model generation method provided in the embodiments of this application;

[0063] Figure 4-a A schematic diagram of the first image provided for an embodiment of this application;

[0064] Figure 4-b for Figure 4-a A schematic diagram of a second image obtained by supplementing the first image with pixels;

[0065] Figure 5 This is a schematic diagram of a first flowchart of an image processing method provided in an embodiment of this application;

[0066] Figure 6This is a second flowchart illustrating the image processing method provided in the embodiments of this application;

[0067] Figure 7 A schematic diagram of the structure of the model generation apparatus provided in the embodiments of this application;

[0068] Figure 8 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application;

[0069] Figure 9 A schematic diagram of the server-side structure provided in an embodiment of this application;

[0070] Figure 10 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application. Detailed Implementation

[0071] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0072] To address the problems in related technologies, embodiments of this application provide a model generation method. This method is applied on the server side. For example... Figure 1 As shown, Figure 1 This is a schematic diagram of a first flowchart of a model generation method provided in an embodiment of this application. The method includes the following steps.

[0073] Step S101: Obtain the first image.

[0074] Step S102: Select multiple resolutions from the preset resolution range of the image processing device as the first resolution.

[0075] Step S103: For each first resolution, perform inference on the first deep learning model corresponding to the first resolution based on the first image to obtain the inference result of the first deep learning model corresponding to the first resolution.

[0076] Step S104: Based on the accuracy of the inference results of the first deep learning model corresponding to each first resolution, determine the first resolution corresponding to the preset number of first deep learning models with the highest accuracy as the optimal resolution of the image processing device.

[0077] Step S105: Generate a second deep learning model for the image processing device based on the first deep learning model corresponding to each optimal resolution.

[0078] pass Figure 1The method described involves the following steps: In the model generation phase, the server selects multiple first resolutions from a preset resolution range of the image processing device. Based on the accuracy of the inference results of the first deep learning model corresponding to each first resolution, the server determines the first resolution corresponding to the preset number of first deep learning models with the highest accuracy as the optimal resolution for the image processing device. Then, based on the first deep learning model corresponding to each optimal resolution, a second deep learning model for the image processing device is generated. In the image processing phase, the image processing device processes the image to be processed based on the third resolution of the image to be processed and the second deep learning model generated in the model generation phase, obtaining the image processing result.

[0079] Compared to related technologies, since the second deep learning model generated during the model generation stage is generated based on the first deep learning model corresponding to a preset number of optimal resolutions, the second deep learning model generated during the model generation stage can support a preset number of optimal resolutions. This allows the second deep learning model to support multiple resolutions, increasing the resolutions supported by the deep learning model.

[0080] Furthermore, in the image processing stage, since the second deep learning model used by the image processing device can support a preset number of optimal resolutions instead of just a fixed resolution, there must be an optimal resolution among the resolutions supported by the second deep learning model that has the smallest error with the resolution of the image to be processed. At this time, using the second deep learning model to process the image to be processed will effectively reduce the error between the resolution supported by the deep learning model and the resolution of the image to be processed, and improve the accuracy of the image processing results.

[0081] The embodiments of this application will be described below through specific examples.

[0082] For step S101 above, namely, obtaining the first image.

[0083] In this step, the server can acquire one or more images as the first image. The specific number of first images acquired is not limited here. For ease of understanding, the following explanation assumes that the number of first images is one.

[0084] In an optional embodiment, the first image described above may include a target object. This target object includes, but is not limited to, people, faces, vehicles, and license plates.

[0085] In this embodiment, the target objects may vary depending on the application scenario of the deep learning model. Therefore, the target objects included in the first image are not specifically limited.

[0086] In an optional embodiment, when acquiring the first image, the server may also acquire the label information of the first image. This label information can be used to indicate the location of the target object in the first image, or the classification result corresponding to the target object, etc.

[0087] For example, when the target object in the first image above is a person, the classification result indicated by the label information can be: elderly, child, teenager, etc.

[0088] For example, when the target object in the first image above is a vehicle, the classification result indicated by the label information can be: sedan, truck, bus, etc.

[0089] In this embodiment, the label information of the first image varies depending on the target object included in the first image. Here, the label information of the first image is not specifically limited. The function of the acquired label information of the first image can be found in the following description, and will not be specifically explained here.

[0090] Regarding step S102 above, multiple resolutions are selected from the preset resolution range of the image processing device as the first resolution.

[0091] In this step, depending on the hardware parameters of the image processing device or the preset requirements of the application scenario, different image processing devices have corresponding preset resolution ranges. The preset resolution range can be represented as the range consisting of the minimum and maximum resolutions that the image processing device can infer. After determining the preset resolution range for each image processing device, the server can select multiple resolutions from that preset resolution range as the first resolution.

[0092] For ease of understanding, let's assume a preset resolution range for a certain image processing device is [100*100, 1000*1000]. The server can select three resolutions from the range [100*100, 1000*1000] as the first resolution. For example, the server can select resolutions of 100*100, 500*500, and 1000*1000 as the first resolutions.

[0093] The number of first resolutions selected by the server can be all resolutions included in the preset resolution range, or it can be a portion of the preset resolution range.

[0094] In one optional embodiment, in addition to having a corresponding preset resolution range, each of the above image processing devices may also have a corresponding preset number, that is, the number of deep learning models of different resolutions that the image processing device can support.

[0095] The number of the aforementioned first resolutions is greater than or equal to the aforementioned preset number. Here, the preset number of the aforementioned first resolutions is not specifically limited.

[0096] The aforementioned image processing device can be an electronic device such as a computer or a video surveillance camera. No specific limitation is made to the image processing device described herein. For ease of understanding, this application embodiment uses only one image processing device as an example for illustration and does not constitute any limitation.

[0097] In one optional embodiment, the preset resolution range of the image processing device can be determined by the preset requirements of the application scenario of the image processing device.

[0098] To facilitate understanding, let's take a mobile phone and a video surveillance camera as image processing devices as examples. Since users have different needs when taking photos with a mobile phone and when taking photos with a video surveillance camera—meaning the resolution of an image captured by a mobile phone can be lower than that of an image captured by a video surveillance camera—the preset resolution ranges for the mobile phone and the video surveillance camera can be determined based on their respective minimum and maximum resolutions. In this case, the determined preset resolution range for the mobile phone can be lower than that for the video surveillance camera.

[0099] In this embodiment of the application, the preset resolution range of the above-mentioned image processing device is not specifically limited.

[0100] For each image processing device, the preset resolution range varies depending on its hardware parameters and the pre-defined requirements of its application scenario. The first resolution selected based on this preset resolution range also varies.

[0101] In an optional embodiment, step S102 above, which involves selecting multiple resolutions from a preset resolution range of the image processing device as the first resolution, can be expressed as:

[0102] The server can select multiple first resolutions from the preset resolution range of the image processing device according to a preset selection algorithm.

[0103] In the embodiments of this application, the preset selection algorithm can be an exhaustive algorithm, a random algorithm, or a uniform distribution algorithm. Here, no specific limitation is made to the preset selection algorithm.

[0104] In another optional embodiment, step S102, which is to select multiple resolutions from the preset resolution range of the image processing device as the first resolution, can be further refined into the following steps, namely step one and step two.

[0105] Step 1: Determine the resolution selection interval based on the image alignment methods supported by the image processing device.

[0106] In this step, for the aforementioned image processing device, when the image supported by the image processing device has a corresponding alignment method, the server can determine the resolution selection interval based on the alignment method of the image supported by the image processing device.

[0107] The image alignment supported by the aforementioned image processing device refers to a parameter of the image resolution displayed by the image processing device (i.e., the width and / or height of the image resolution) being an integer multiple of a certain number. Taking 32-bit alignment as an example, the image resolution displayed by the image processing device can be 1920*1080, 3840*2160, etc. Where 1920 = 32*60, 3840 = 32*128.

[0108] When the image processing device supports 32-bit alignment, the server can determine the resolution selection interval as a multiple of 32, such as 160 or 320.

[0109] In the embodiments of this application, the resolution selection interval is determined differently depending on the alignment method of the image supported by the image processing device. Here, the resolution selection interval is not specifically limited.

[0110] Step two: Select multiple resolutions from the preset resolution range of the image processing device according to the resolution selection interval, and use them as the first resolution.

[0111] Through steps one and two above, when the image supported by the image processing device has a corresponding alignment method, the resolution selection interval is determined according to the alignment method, and the first resolution is selected according to the resolution selection interval. This allows the selected first resolution to match the alignment method of the image supported by the image processing device, thereby improving the accuracy of the selected first resolution.

[0112] Furthermore, since each first resolution is selected at the same resolution selection interval, the selected first resolutions are evenly distributed within the preset resolution range of the aforementioned image processing device. This effectively ensures the uniform distribution of the selected first resolutions, thereby making the optimal resolution determined later more representative and improving the accuracy of the determined optimal resolution, which in turn improves the accuracy of the generated model.

[0113] For step S103 above, that is, for each first resolution, the first deep learning model corresponding to the first resolution is inferred based on the first image to obtain the inference result of the first deep learning model corresponding to the first resolution.

[0114] In this step, for each first resolution, the server can use the aforementioned first image to perform forward inference on the first deep learning model corresponding to that first resolution, obtaining the inference result of the first deep learning model corresponding to that first resolution. Depending on the model structure and purpose of each first deep learning model, the forward inference process and inference result of each first deep learning model will differ. Here, no specific limitations are made on the forward inference process and inference result of each first deep learning model.

[0115] The first deep learning model mentioned above can be an object detection model, or an object classification model, etc.

[0116] In addition, the aforementioned first deep learning model can be a deep learning model pre-trained on the server side, or it can be a deep learning model that has not been trained.

[0117] In the embodiments of this application, the first deep learning model is not specifically limited.

[0118] In an optional embodiment, when the first deep learning model is a pre-trained deep learning model, the first deep learning models corresponding to each resolution are the same in terms of model structure and model parameters, except that the image resolutions supported by each first deep learning model are different.

[0119] In another optional embodiment, when the first deep learning model is an untrained deep learning model, the server trains the first deep learning model corresponding to each first resolution, and uses the first image to perform inference on the trained first deep learning model to obtain the inference result of the first deep learning model corresponding to each first resolution. In this embodiment, the resolutions, model parameters, etc., supported by the first deep learning model corresponding to each first resolution obtained by training can be different.

[0120] The training process of each of the above-mentioned first deep learning models can refer to the training methods of related models in related technologies, and will not be specifically explained here.

[0121] In this embodiment, the first deep learning model described above can be trained or represented based on a deep learning framework, such as a convolutional neural network framework (Caffe), TensorFlow, or PyTorch. TensorFlow is a deep learning framework, and PyTorch is a machine learning library in the Python programming language. No specific limitations are imposed on the first deep learning model described above.

[0122] Regarding step S104 above, that is, based on the accuracy of the inference results of the first deep learning model corresponding to each first resolution, the first resolution corresponding to the preset number of first deep learning models with the highest accuracy is determined as the optimal resolution of the image processing device.

[0123] In this step, for each first resolution, after determining the inference result of the first deep learning model corresponding to the first resolution, the server can determine the accuracy of the inference result of the first deep learning model corresponding to the first resolution, and determine the first resolution corresponding to the preset number of first deep learning models with the highest accuracy as the optimal resolution of the image processing device.

[0124] For ease of understanding, we will take a first resolution of 5 (i.e., resolution 1 to resolution 5) as an example.

[0125] Based on the inference results of the first deep learning model corresponding to each resolution, the server determines the accuracy of the first deep learning model for each resolution from highest to lowest as follows: resolution 5, resolution 4, resolution 3, resolution 2, and resolution 1. If the preset number of the above image processing devices is 3, then the server can determine resolution 5, resolution 4, and resolution 3 as the optimal resolutions for that image processing device.

[0126] In an optional embodiment, the aforementioned preset quantity may be determined based on the hardware parameters of the image processing device. These hardware parameters include, but are not limited to, the memory and computing resources of the image processing device.

[0127] To facilitate understanding, let's take a mobile phone as an example for image processing. While determining the preset resolution range corresponding to the mobile phone, we can also determine the number of first resolutions selected from that preset resolution range. At this point, the server can adjust this number based on the phone's hardware parameters to obtain the aforementioned preset number. For example, if the phone has a large amount of memory, the preset number can be increased; if the phone has a small amount of memory, the preset number can be decreased.

[0128] In this embodiment of the application, the preset number of image processing devices is not specifically limited.

[0129] In an optional embodiment, when the first deep learning model is an untrained deep learning model, in order to improve the accuracy of the determined optimal resolution, the optimal resolution can be the first resolution corresponding to a preset number of first deep learning models that have the highest accuracy among multiple first resolutions and whose accuracy is greater than a preset accuracy threshold. The preset accuracy threshold can be set according to requirements or user experience, and here, no specific limitation is made on the preset accuracy threshold.

[0130] In an optional embodiment, when the first deep learning model is an untrained deep learning model, if the plurality of first resolutions does not include the first resolutions corresponding to a preset number of first deep learning models with the highest accuracy and an accuracy greater than a preset accuracy threshold, the server can adjust the preset resolution range and the preset number. The specific method for adjusting the preset resolution range and the preset number is not described here.

[0131] In this embodiment, the preset resolution range and preset quantity can be determined not only based on the preset requirements of the image processing device application scenario or the hardware parameters of the image processing device, but also through other methods. For example, they can be set based on user experience values. Here, the method for determining the preset resolution range and preset quantity is not specifically limited.

[0132] For step S105 above, that is, generating a second deep learning model for the image processing device based on the first deep learning model corresponding to each optimal resolution.

[0133] In this step, after determining the preset number of optimal resolutions corresponding to the aforementioned image processing device, the server can obtain a first deep learning model corresponding to each optimal resolution, and generate a second deep learning model based on the first deep learning model corresponding to each optimal resolution. This second deep learning model is the deep learning model used by the image processing device when processing the image to be processed.

[0134] In an optional embodiment, step S105 above, which generates a second deep learning model for the image processing device based on the first deep learning model corresponding to each optimal resolution, may include the following steps, namely step one and step two.

[0135] Step 1: For each optimal resolution, compile the first deep learning model corresponding to that optimal resolution according to the model format that the image processing device can recognize, to obtain the third deep learning model of the image processing device.

[0136] In this embodiment, the first deep learning model is trained on the server side; however, its application is performed on an image processing device. Different image processing devices have different hardware parameters and can recognize different model formats. Therefore, after determining each optimal resolution, the server can compile the first deep learning model corresponding to each optimal resolution according to the model formats recognizable by the image processing device, thus obtaining a third deep learning model corresponding to each optimal resolution. Each third deep learning model can be recognized by the image processing device.

[0137] In an optional embodiment, when compiling each first deep learning model, the server can utilize compilation tools from related technologies, such as a platform compiler, to complete the compilation process of the first deep learning model. The specific details of the compilation process described above will not be provided here.

[0138] Step two: Package each third deep learning model to obtain the second deep learning model.

[0139] In this step, the server can package a preset number of third deep learning models into a single deep learning model to obtain the second deep learning model of the aforementioned image processing device.

[0140] Through steps one and two above, the server can generate a second deep learning model based on a predetermined number of first deep learning models corresponding to optimal resolutions. This enables the image processing device to process images of various resolutions when using the second deep learning model, thereby improving the accuracy of the image processing results.

[0141] In an optional embodiment, when the first deep learning model is an object detection model, according to Figure 1 The method shown in this application embodiment also provides a model generation method. For example... Figure 2 As shown, Figure 2 This is a second flowchart illustrating the model generation method provided in an embodiment of this application. The method includes the following steps.

[0142] Step S201: Obtain the first image.

[0143] Step S202: Select multiple resolutions from the preset resolution range of the image processing device as the first resolution.

[0144] The steps S201-S202 described above are the same as the steps S101-S102 described above.

[0145] Step S203: For each first resolution, the target object in the first image is detected using the first deep learning model corresponding to the first resolution, and the detection result of the first deep learning model corresponding to the first resolution is obtained.

[0146] In this step, for each resolution, the server can use the first deep learning model corresponding to the first resolution to detect the target object in the first image, determine whether the target object exists in the first image and the location of the target object, and obtain the detection result of the first deep learning model corresponding to the first resolution. The detection result is the inference result of the first deep learning model corresponding to the first resolution.

[0147] The above detection results can indicate whether a target object exists in the first image, or the location information of the target object in the first image. No specific limitations are made regarding the above detection results here.

[0148] Step S204: Based on the detection results of the first depth model corresponding to each first resolution, determine the detection rate and / or IOU index of the target object in the first image by the first deep learning model corresponding to the first resolution.

[0149] In this step, when the first deep learning model is trained using supervised learning, the server can obtain the label information of the first image when acquiring it. This label information can indicate whether a target object exists in the first image, or, if so, the location of the target object. For each first resolution corresponding to the first deep learning model, after the server uses that model to perform target detection on the first image and obtains the detection result, it can calculate the detection rate and / or IOU index of the target object in the first image for that first resolution based on the detection result and the label information.

[0150] The detection rate and IOU metrics mentioned above represent the accuracy of the detection results obtained by each first deep learning model in detecting objects in the first image, that is, the accuracy of the inference results of each first deep learning model. Higher detection rates and IOU metrics indicate higher detection accuracy and higher inference accuracy of the first deep learning model; conversely, lower detection rates and IOU metrics indicate lower detection accuracy and higher inference accuracy of the first deep learning model. The specific calculation methods for the detection rate and IOU metrics are not explained here.

[0151] In this embodiment, the first deep learning model can be trained not only through supervised learning but also through unsupervised learning. In this case, for each first resolution, the server can determine the accuracy of the inference result of the first deep learning model based on other metrics for object detection in the first image. These other metrics may include the number of detection boxes indicating the location of the target object in the detection results, or the detection rate, etc. No specific limitations are imposed on these other metrics.

[0152] Step S205: The first resolution corresponding to the preset number of first deep learning models with the highest detection rate and / or IOU index is determined as the optimal resolution of the image processing device.

[0153] Steps S204-S205 above are a refinement of step S104 above.

[0154] When the first deep learning model is an object detection model, the server can determine the first deep learning model corresponding to a preset number of optimal resolutions based on the accuracy of the detection results corresponding to each first deep learning model through steps S203-S205, thereby generating the corresponding second deep learning model, so that the image processing device can directly use the second deep learning model to perform object detection.

[0155] Step S206: Generate a second deep learning model for the image processing device based on the first deep learning model corresponding to each optimal resolution.

[0156] The above step S206 is the same as the above step S105.

[0157] In the embodiments of this application, the first deep learning model described above can be, in addition to the target detection model used for target detection, a target recognition model used for target identification, or a target classification model used for target classification. The specific application of the first deep learning model is not limited here.

[0158] In an optional embodiment, according to Figure 2 The method shown in this application embodiment also provides a model generation method. For example... Figure 3 As shown, Figure 3 This is a third flowchart illustrating the model generation method provided in this application embodiment. In this method, step S203 is further refined into the following steps, namely steps S2031-S2032.

[0159] Step S2031: For each first resolution, if the first resolution is greater than the second resolution of the first image, then add pixels with preset values ​​to the right side of the right boundary and the bottom side of the bottom boundary of the first image to obtain a second image with the resolution of the first resolution.

[0160] In this embodiment, for each first resolution, since the first resolution does not match the second resolution of the first image, the server cannot use the first deep learning model corresponding to the first resolution to perform inference on the first image. Therefore, when the server determines that the first resolution is greater than the second resolution of the first image, it can adjust the resolution of the first image so that the adjusted resolution of the first image is the first resolution, thus obtaining the second image.

[0161] In one optional embodiment, for each first resolution, when the first resolution is greater than the second resolution of the first image, the server can supplement the right side of the right boundary and the bottom side of the bottom boundary of the first image with pixels of a preset value to obtain a second image with the resolution of the first resolution.

[0162] For ease of understanding, combined with Figure 4-a and Figure 4-b The above-mentioned pixels are further explained. Figure 4-a A schematic diagram of the first image provided in an embodiment of this application. Figure 4-b for Figure 4-a A schematic diagram of a second image obtained by supplementing the first image with pixels.

[0163] exist Figure 4-a The first image shown contains 4*4 = 16 pixels, meaning its second resolution can be represented as 4*4. Now, assume the first resolution is 5*5. Since the first resolution is greater than the second resolution, the server can add pixels with a value of 0 to the right of the right boundary and the bottom of the bottom boundary of the first image, resulting in the image shown below. Figure 4-b The second image shown has a resolution of 5*5.

[0164] In an optional embodiment, the preset value can be 0. The preset value can be set according to user needs or the usage scenario of the image processing device, and there is no specific limitation on the preset value.

[0165] Step S2032: Using the first deep learning model corresponding to the first resolution, the target object in the second image is detected to obtain the detection result of the first deep learning model corresponding to the first resolution.

[0166] In this embodiment, by supplementing pixels to the right of the right boundary and the bottom of the bottom boundary of the first image, the image content corresponding to the first image can always be located in the upper left position of the second image. This makes it easier to accurately locate the image content corresponding to the first image when performing inference on the second image later, thereby reducing the computational load of the deep learning model inference process and saving computational resources.

[0167] Based on the same inventive concept, and according to the model generation method provided in the embodiments of this application above, this application also provides an image processing method. This method is applied to the aforementioned image processing device. For example... Figure 5 As shown, Figure 5 This is a schematic flowchart of a first image processing method provided in an embodiment of this application. The method includes the following steps.

[0168] Step S501: Obtain the second deep learning model of the image processing device from the server. The second deep learning model is generated based on the first deep learning model corresponding to a preset number of optimal resolutions of the image processing device. The preset number of optimal resolutions are determined based on the accuracy of the inference results of the first deep learning models corresponding to multiple first resolutions in the preset resolution range of the image processing device.

[0169] In one optional embodiment, after generating the second deep learning model for the image processing device, the server can distribute the second deep learning model to the image processing device. For example, the server can distribute the second deep learning model to the image processing device through a release mechanism.

[0170] In another optional embodiment, after generating the second deep learning model for the image processing device, the server can send a notification message to the image processing device. Upon receiving the notification message, the image processing device can obtain the second deep learning model from the server.

[0171] Here, no specific limitation is made on the method by which the image processing device acquires the second deep learning model.

[0172] In the embodiments of this application, the image processing device mentioned above includes, but is not limited to, a computer, a video surveillance camera, etc.

[0173] In one optional embodiment, the second deep learning model includes a preset number of third deep learning models corresponding to optimal resolutions, and each third deep learning model corresponding to an optimal resolution is compiled from the first deep learning model corresponding to that optimal resolution.

[0174] Step S502: Obtain the image to be processed.

[0175] In an optional embodiment, when the image processing device includes an image acquisition unit, such as a camera, the image to be processed can be an image acquired in real time by the image acquisition unit in the image processing device.

[0176] In another optional embodiment, the image to be processed may also be an image transmitted in real time by an image acquisition device that is communicatively connected to the image processing device.

[0177] Step S503: Based on the third resolution of the image to be processed, the image to be processed is processed using the second deep learning model to obtain the image processing result.

[0178] In this step, since the second deep learning model includes a preset number of third deep learning models corresponding to the optimal resolution, when processing the image to be processed, the image processing device can select the third deep learning model to process the image from the third deep learning models included in the second deep learning model according to the third resolution of the image to be processed, and use the third deep learning model to process the image to obtain the image processing result.

[0179] In this application embodiment, the image processing results vary depending on the purpose of the second deep learning model. For example, when the second deep learning model is an object detection model, the image processing results can indicate whether a target object exists in the image to be processed, and if so, the location of the target object. As another example, when the second deep learning model is an object classification model, the image processing results can indicate the classification result corresponding to the target object in the image to be processed. Here, no specific limitation is made to the image processing results.

[0180] pass Figure 5 The method shown describes an image processing device that, in the image processing stage, processes the image to be processed based on the third resolution of the image to be processed and the second deep learning model generated in the model generation stage to obtain the image processing result.

[0181] Compared to related technologies, since the second deep learning model generated during the model generation stage is generated based on the first deep learning model corresponding to a preset number of optimal resolutions, the second deep learning model generated during the model generation stage can support a preset number of optimal resolutions. This allows the second deep learning model to support multiple resolutions, increasing the resolutions supported by the deep learning model.

[0182] Furthermore, in the image processing stage, since the second deep learning model used by the image processing device can support a preset number of optimal resolutions instead of just a fixed resolution, there must be an optimal resolution among the resolutions supported by the second deep learning model that has the smallest error with the resolution of the image to be processed. At this time, using the second deep learning model to process the image to be processed will effectively reduce the error between the resolution supported by the deep learning model and the resolution of the image to be processed, and improve the accuracy of the image processing results.

[0183] In an optional embodiment, according to Figure 5 The method shown in this application embodiment also provides an image processing method. For example... Figure 6 As shown, Figure 6 This is a second flowchart illustrating the image processing method provided in an embodiment of this application. In this method, step S503 is further refined into the following steps S5031-S5033.

[0184] Step S5031: Based on the optimal resolution corresponding to each of the third deep learning models in the second deep learning model, determine the optimal resolution with the smallest error between the resolution and the third resolution of the image to be processed as the target resolution, and the target resolution is greater than or equal to the third resolution.

[0185] In this step, after acquiring the image to be processed, the image processing device compares the third resolution of the image to be processed with the optimal resolution corresponding to each of the third deep learning models included in the second deep learning model. The optimal resolution, which is greater than or equal to the third resolution and has the smallest error with the third resolution, is then determined as the target resolution.

[0186] For ease of understanding, let's take an example where the third resolution of the image to be processed is 150*150. Now assume that the optimal resolutions of the third deep learning models included in the second deep learning model are 100*100, 200*200, and 300*300. In this case, the image processing device can determine that the target resolution corresponding to the third resolution of the image to be processed is 200*200.

[0187] Step S5032: If the target resolution is greater than the third resolution, then add pixels with preset values ​​to the right side of the right boundary and the bottom side of the bottom boundary of the image to be processed to obtain a third image with the target resolution.

[0188] In this step, when the target resolution is greater than the third resolution, in order to facilitate the use of the third deep learning model corresponding to the target resolution to process the image to be processed, the image processing device can adjust the resolution of the image to be processed, that is, adjust the resolution of the image to be processed from the third resolution to the target resolution to obtain the third image.

[0189] For ease of understanding, we will still use the example of the third resolution of the image to be detected being 150*150 and the target resolution being 200*200.

[0190] Since the target resolution is greater than the third resolution of the image to be processed, the image processing device needs to adjust the resolution of the image to be processed. Specifically, the image processing device adds 200*200 - 150*150 = 17500 pixels with preset pixel values ​​to the right of the right boundary and to the bottom of the bottom boundary of the image to be processed, resulting in a third image with a resolution of 200*200.

[0191] The method described above for adjusting the resolution of the image to be processed to the target resolution is the same as the method described above for adjusting the resolution of the first image to the first resolution. The specific method for adjusting the resolution of the image to be processed will not be explained here.

[0192] Step S5033: Use the third deep learning model corresponding to the target resolution in the second deep learning model to process the third image and obtain the image processing result.

[0193] In this step, after adjusting the resolution of the image to be processed to the target resolution to obtain the third image, the image processing device can use the third deep learning model corresponding to the target resolution in the second deep learning model to process the third image and obtain the image processing result.

[0194] In an optional embodiment, when the target resolution is equal to the third resolution mentioned above, the image processing device can directly use the third deep learning model corresponding to the target resolution in the second deep learning model to process the third image and obtain the image processing result.

[0195] In related technologies, since the resolution supported by a deep learning model is unique and fixed, when the resolution of the image to be processed is much smaller than the resolution supported by the deep learning model, the image processing device adds a large number of pixels to the image to be processed, introducing more noise data, which will seriously affect the accuracy of the image processing results.

[0196] Compared to related technologies, through the above steps S5031-S5033, the second deep learning model includes a third deep learning model corresponding to a preset number of optimal resolutions. When processing the image to be processed, since the target resolution is the optimal resolution among the preset number of optimal resolutions that is greater than the third resolution and has the smallest error with the third resolution, the third image obtained by supplementing pixels based on the target resolution introduces less noise data, reducing the impact of noise data on the accuracy of image processing results and improving the accuracy of image processing results.

[0197] Based on the same inventive concept, and according to the model generation method provided in the above embodiments of this application, this application also provides a model generation apparatus. This apparatus is applied to the aforementioned server. For example... Figure 7 As shown, Figure 7 This is a schematic diagram of a model generation apparatus provided in an embodiment of this application. The apparatus includes the following modules.

[0198] The first acquisition module 701 is used to acquire the first image;

[0199] The selection module 702 is used to select multiple resolutions from the preset resolution range of the image processing device as the first resolution;

[0200] The inference module 703 is used to perform inference on the first deep learning model corresponding to each first resolution based on the first image, and obtain the inference result of the first deep learning model corresponding to the first resolution.

[0201] The determining module 704 is used to determine the first resolution corresponding to the preset number of first deep learning models with the highest accuracy as the optimal resolution of the image processing device based on the accuracy of the inference results of the first deep learning model corresponding to each first resolution.

[0202] The generation module 705 is used to generate a second deep learning model for the image processing device based on the first deep learning model corresponding to each optimal resolution.

[0203] Optionally, the selection module 702 can be specifically used to determine the resolution selection interval according to the image alignment method supported by the image processing device; and select multiple resolutions from the preset resolution range of the image processing device according to the resolution selection interval as the first resolution.

[0204] Optionally, the inference module 703 is specifically used to detect the target object in the first image for each first resolution by using the first deep learning model corresponding to the first resolution if the first deep learning model is an object detection model, and to obtain the detection result of the first deep learning model corresponding to the first resolution.

[0205] The aforementioned determining module 704 can be specifically used to determine the detection rate and / or IOU index of the first deep learning model for the target object in the first image based on the detection result of the first deep learning model corresponding to each first resolution if the first deep learning model is a target detection model; and to determine the first resolution corresponding to the preset number of first deep learning models with the highest detection rate and / or IOU index as the optimal resolution of the image processing device.

[0206] Optionally, the inference module 703 can be used to, for each first resolution, if the first resolution is greater than the second resolution of the first image, supplement the right side of the right boundary and the bottom side of the lower boundary of the first image with pixels of preset values ​​to obtain a second image with the resolution of the first resolution; and use the first deep learning model corresponding to the first resolution to detect the target objects in the second image to obtain the detection result of the first deep learning model corresponding to the first resolution.

[0207] Optionally, the aforementioned generation module 705 can be used to compile the first deep learning model corresponding to each optimal resolution according to the model format that the image processing device can recognize, to obtain the third deep learning model of the image processing device; and package each third deep learning model to obtain the second deep learning model.

[0208] Based on the same inventive concept, and according to the image processing method provided in the above embodiments of this application, the embodiments of this application also provide an image processing apparatus. For example... Figure 8 As shown, Figure 8 This is a schematic diagram of an image processing apparatus provided in an embodiment of this application. The apparatus includes the following modules.

[0209] The second acquisition module 801 is used to acquire the second deep learning model of the image processing device from the server. The second deep learning model is generated based on the first deep learning model corresponding to a preset number of optimal resolutions of the image processing device. The preset number of optimal resolutions are determined based on the accuracy of the inference results of the first deep learning models corresponding to multiple first resolutions in the preset resolution range of the image processing device.

[0210] The third acquisition module 802 is used to acquire the image to be processed;

[0211] The processing module 803 is used to process the image to be processed based on the third resolution of the image to be processed and the second deep learning model to obtain the image processing result.

[0212] Optionally, the second deep learning model mentioned above includes a preset number of third deep learning models corresponding to the optimal resolution, and each third deep learning model corresponding to the optimal resolution is compiled from the first deep learning model corresponding to the optimal resolution.

[0213] The aforementioned processing module 803 can be specifically used to determine the optimal resolution with the smallest error between the resolution and the third resolution of the image to be processed, based on the optimal resolution corresponding to each third deep learning model in the second deep learning model, and the target resolution is greater than or equal to the third resolution; if the target resolution is greater than the third resolution, then pixels with preset values ​​are added to the right side of the right boundary and the bottom side of the bottom boundary of the image to be processed to obtain a third image with the target resolution; the third image is processed using the third deep learning model corresponding to the target resolution in the second deep learning model to obtain the image processing result.

[0214] In the model generation stage, the server selects multiple first resolutions from a preset resolution range of the image processing device. Based on the accuracy of the inference results of the first deep learning model corresponding to each first resolution, the server determines the first resolution corresponding to the preset number of first deep learning models with the highest accuracy as the optimal resolution for the image processing device. Then, based on the first deep learning model corresponding to each optimal resolution, a second deep learning model for the image processing device is generated. In the image processing stage, the image processing device processes the image to be processed based on the third resolution of the image to be processed and the second deep learning model generated in the model generation stage to obtain the image processing result.

[0215] Compared to related technologies, since the second deep learning model generated during the model generation stage is generated based on the first deep learning model corresponding to a preset number of optimal resolutions, the second deep learning model generated during the model generation stage can support a preset number of optimal resolutions. This allows the second deep learning model to support multiple resolutions, increasing the resolutions supported by the deep learning model.

[0216] Furthermore, in the image processing stage, since the second deep learning model used by the image processing device can support a preset number of optimal resolutions instead of just a fixed resolution, there must be an optimal resolution among the resolutions supported by the second deep learning model that has the smallest error with the resolution of the image to be processed. At this time, using the second deep learning model to process the image to be processed will effectively reduce the error between the resolution supported by the deep learning model and the resolution of the image to be processed, and improve the accuracy of the image processing results.

[0217] Based on the same inventive concept, and according to the model generation method provided in the above embodiments of this application, this application also provides an electronic device, which is the aforementioned server-side device, such as... Figure 9 As shown, it includes a processor 901, a communication interface 902, a memory 903, and a communication bus 904, wherein the processor 901, the communication interface 902, and the memory 903 communicate with each other through the communication bus 904.

[0218] Memory 903 is used to store computer programs;

[0219] When processor 901 executes a program stored in memory 903, it performs the following steps:

[0220] Get the first image;

[0221] Multiple resolutions are selected from the preset resolution range of the image processing device as the first resolution;

[0222] For each first resolution, inference is performed on the first deep learning model corresponding to the first resolution based on the first image to obtain the inference result of the first deep learning model corresponding to the first resolution;

[0223] Based on the accuracy of the inference results of the first deep learning model corresponding to each first resolution, the first resolution corresponding to the preset number of first deep learning models with the highest accuracy is determined as the optimal resolution of the image processing device.

[0224] A second deep learning model for the image processing device is generated based on the first deep learning model corresponding to each optimal resolution.

[0225] Based on the same inventive concept, and according to the image processing method provided in the above embodiments of this application, this application also provides an electronic device, which is the aforementioned image processing device, such as... Figure 10 As shown, it includes a processor 1001, a communication interface 1002, a memory 1003, and a communication bus 1004, wherein the processor 1001, the communication interface 1002, and the memory 1003 communicate with each other through the communication bus 1004.

[0226] Memory 1003 is used to store computer programs;

[0227] When processor 1001 executes a program stored in memory 1003, it performs the following steps:

[0228] The second deep learning model of the image processing device is obtained from the server. The second deep learning model is generated based on the first deep learning model corresponding to a preset number of optimal resolutions of the image processing device. The preset number of optimal resolutions are determined based on the accuracy of the inference results of the first deep learning models corresponding to multiple first resolutions in the preset resolution range of the image processing device.

[0229] Obtain the image to be processed;

[0230] Based on the third resolution of the image to be processed, the image is processed using a second deep learning model to obtain the image processing result.

[0231] In the electronic device provided by this application embodiment, during the model generation stage, the server selects multiple first resolutions from a preset resolution range of the image processing device. Based on the accuracy of the inference results of the first deep learning model corresponding to each first resolution, the first resolution corresponding to the preset number of first deep learning models with the highest accuracy is determined as the optimal resolution corresponding to the image processing device. Then, based on the first deep learning model corresponding to each optimal resolution, a second deep learning model for the image processing device is generated. During the image processing stage, the image processing device processes the image to be processed based on the third resolution of the image to be processed and the second deep learning model generated during the model generation stage to obtain the image processing result.

[0232] Compared to related technologies, since the second deep learning model generated during the model generation stage is generated based on the first deep learning model corresponding to a preset number of optimal resolutions, the second deep learning model generated during the model generation stage can support a preset number of optimal resolutions. This allows the second deep learning model to support multiple resolutions, increasing the resolutions supported by the deep learning model.

[0233] Furthermore, in the image processing stage, since the second deep learning model used by the image processing device can support a preset number of optimal resolutions instead of just a fixed resolution, there must be an optimal resolution among the resolutions supported by the second deep learning model that has the smallest error with the resolution of the image to be processed. At this time, using the second deep learning model to process the image to be processed will effectively reduce the error between the resolution supported by the deep learning model and the resolution of the image to be processed, and improve the accuracy of the image processing results.

[0234] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0235] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0236] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0237] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0238] Based on the same inventive concept, and according to the model generation method provided in the above embodiments of this application, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above model generation methods.

[0239] Based on the same inventive concept, and according to the model generation method provided in the above embodiments of this application, this application also provides a computer program product containing instructions, which, when run on a computer, causes the computer to execute any of the model generation methods in the above embodiments.

[0240] Based on the same inventive concept, and according to the image processing method provided in the above embodiments of this application, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above image processing methods.

[0241] Based on the same inventive concept, and according to the image processing methods provided in the above embodiments of this application, this application also provides a computer program product containing instructions, which, when run on a computer, causes the computer to execute any of the image processing methods in the above embodiments.

[0242] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0243] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0244] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, embodiments such as apparatuses, electronic devices, computer-readable storage media, and computer program products are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0245] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A model generation method, characterized in that, Applied to the server side, the method includes: Get the first image; Multiple resolutions are selected from the preset resolution range of the image processing device as the first resolution; For each first resolution, inference is performed on the first deep learning model corresponding to the first resolution based on the first image to obtain the inference result of the first deep learning model corresponding to the first resolution; Based on the accuracy of the inference results of the first deep learning model corresponding to each first resolution, the first resolution corresponding to the preset number of first deep learning models with the highest accuracy is determined as the optimal resolution of the image processing device. Based on the first deep learning model corresponding to each optimal resolution, a second deep learning model for the image processing device is generated. After acquiring the image to be processed, the image processing device determines the optimal resolution with the smallest error between its resolution and the third resolution of the image to be processed, based on the optimal resolution corresponding to each third deep learning model in the second deep learning model. The target resolution is greater than or equal to the third resolution. If the target resolution is greater than the third resolution, pixels with preset values ​​are added to the right side of the right boundary and the bottom side of the bottom boundary of the image to be processed, resulting in a third image with the target resolution. The third image is then processed using the third deep learning model corresponding to the target resolution in the second deep learning model to obtain the image processing result. The second deep learning model includes the preset number of third deep learning models corresponding to optimal resolutions, and each third deep learning model corresponding to an optimal resolution is compiled from the first deep learning model corresponding to that optimal resolution.

2. The method according to claim 1, characterized in that, The step of selecting multiple resolutions from a preset resolution range of the image processing device as the first resolution includes: The resolution selection interval is determined based on the image alignment method supported by the image processing device. According to the resolution selection interval, multiple resolutions are selected from the preset resolution range of the image processing device as the first resolution.

3. The method according to claim 1, characterized in that, If the first deep learning model is an object detection model, then the step of performing inference on the first deep learning model corresponding to each first resolution based on the first image to obtain the inference result of the first deep learning model corresponding to the first resolution includes: For each first resolution, the target object in the first image is detected using the first deep learning model corresponding to that first resolution, and the detection result of the first deep learning model corresponding to that first resolution is obtained. The step of determining the first resolution corresponding to a preset number of first deep learning models with the highest accuracy as the optimal resolution of the image processing device based on the accuracy of the inference results of the first deep learning model corresponding to each first resolution includes: Based on the detection results of the first deep learning model corresponding to each first resolution, determine the detection rate and / or intersection-union ratio (IOU) of the first deep learning model for the target object in the first image for that first resolution; The first resolution corresponding to the highest preset number of first deep learning models with the highest detection rate and / or IOU index is determined as the optimal resolution of the image processing device.

4. The method according to claim 3, characterized in that, The step of detecting objects in the first image using a first deep learning model corresponding to each first resolution, and obtaining the detection result of the first deep learning model corresponding to the first resolution, includes: For each first resolution, if the first resolution is greater than the second resolution of the first image, then pixels with preset values ​​are added to the right side of the right boundary and the bottom side of the bottom boundary of the first image to obtain a second image with the resolution of the first resolution. Using the first deep learning model corresponding to the first resolution, the target objects in the second image are detected, and the detection results of the first deep learning model corresponding to the first resolution are obtained.

5. The method according to claim 1, characterized in that, The step of generating a second deep learning model for the image processing device based on a first deep learning model corresponding to each optimal resolution includes: For each optimal resolution, the first deep learning model corresponding to the optimal resolution is compiled according to the model format that the image processing device can recognize, to obtain the third deep learning model of the image processing device. By packaging the third deep learning models, we obtain the second deep learning model.

6. An image processing method, characterized in that, Applied to an image processing device, the method includes: The second deep learning model of the image processing device is obtained from the server. The second deep learning model is generated based on the first deep learning model corresponding to a preset number of optimal resolutions of the image processing device. The preset number of optimal resolutions are determined based on the accuracy of the inference results obtained by inferring the first image to multiple first resolutions corresponding to the first deep learning models in the preset resolution range of the image processing device. The second deep learning model includes the third deep learning model corresponding to the preset number of optimal resolutions. The third deep learning model corresponding to each optimal resolution is compiled from the first deep learning model corresponding to that optimal resolution. Obtain the image to be processed; Based on the optimal resolution corresponding to each of the third deep learning models in the second deep learning model, the optimal resolution with the smallest error between the resolution and the third resolution of the image to be processed is determined as the target resolution, and the target resolution is greater than or equal to the third resolution; If the target resolution is greater than the third resolution, then pixels with preset values ​​are added to the right side of the right boundary and the bottom side of the bottom boundary of the image to be processed to obtain a third image with the target resolution. The third image is processed using the third deep learning model corresponding to the target resolution in the second deep learning model to obtain the image processing result.

7. A model generation apparatus, characterized in that, The device, applied to the server side, includes: The first acquisition module is used to acquire the first image; The selection module is used to select multiple resolutions from the preset resolution range of the image processing device as the first resolution; The inference module is used to perform inference on the first deep learning model corresponding to each first resolution based on the first image, and obtain the inference result of the first deep learning model corresponding to the first resolution. The determining module is used to determine the first resolution corresponding to the preset number of first deep learning models with the highest accuracy as the optimal resolution of the image processing device based on the accuracy of the inference results of the first deep learning model corresponding to each first resolution. The generation module is used to generate a second deep learning model for the image processing device based on the first deep learning model corresponding to each optimal resolution. This allows the image processing device, after acquiring the image to be processed, to determine the optimal resolution with the smallest error between its resolution and the third resolution of the image to be processed, based on the optimal resolutions corresponding to the third deep learning models in the second deep learning model. The target resolution is greater than or equal to the third resolution. If the target resolution is greater than the third resolution, pixels with preset values ​​are added to the right side of the right boundary and the bottom side of the lower boundary of the image to be processed to obtain a third image with the target resolution. The third image is then processed using the third deep learning model corresponding to the target resolution in the second deep learning model to obtain an image processing result. The second deep learning model includes the preset number of third deep learning models corresponding to optimal resolutions, and each third deep learning model corresponding to an optimal resolution is compiled from the first deep learning model corresponding to that optimal resolution.

8. An image processing apparatus, characterized in that, Applied to an image processing device, the device includes: The second acquisition module is used to acquire the second deep learning model of the image processing device from the server. The second deep learning model is generated based on the first deep learning model corresponding to a preset number of optimal resolutions of the image processing device. The preset number of optimal resolutions are determined based on the accuracy of the inference results obtained by inferring the first image to multiple first resolutions corresponding to the first deep learning models in the preset resolution range of the image processing device. The second deep learning model includes the third deep learning model corresponding to the preset number of optimal resolutions. The third deep learning model corresponding to each optimal resolution is compiled from the first deep learning model corresponding to that optimal resolution. The third acquisition module is used to acquire the image to be processed; The processing module is configured to determine the optimal resolution with the smallest error between the resolution and the third resolution of the image to be processed as the target resolution, based on the optimal resolution corresponding to each third deep learning model in the second deep learning model, wherein the target resolution is greater than or equal to the third resolution; if the target resolution is greater than the third resolution, then pixels with preset values ​​are added to the right side of the right boundary and the bottom side of the bottom boundary of the image to be processed to obtain a third image with the target resolution; and the third image is processed using the third deep learning model corresponding to the target resolution in the second deep learning model to obtain an image processing result.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1-5 or claim 6.

Images