Image processing model optimization method and device, and storage medium

Abstract

The application discloses an image processing model optimization method and device and a storage medium. The method comprises the following steps: acquiring a user image; performing desensitization processing on the user image to obtain first desensitization data corresponding to the user image; extracting marked data corresponding to the first desensitization data from the user image, wherein the marked data is second desensitization data containing marked information; and optimizing a current image processing model based on the first desensitization data and the marked data to obtain an optimized image processing model. In the above scheme, the image processing model is optimized based on the first desensitization data, so that the image processing model is better applicable to the current application scenario, and the accuracy of the image processing model is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image processing model optimization method, device and storage medium. Background Art

[0002] With the continuous development of science and technology, the application of image processing models is becoming more and more extensive. For example, in the field of human-computer interaction, image processing models can recognize users' gestures, expressions, and actions to achieve specific operations and functions.

[0003] To protect users' personal information, existing technologies require desensitizing user images during the training and use of image processing models. However, since desensitized data cannot be identified by any means, it cannot be used to optimize image processing models. Summary of the Invention

[0004] The embodiments of the present application provide an image processing model optimization method, device and storage medium.

[0005] In a first aspect, the present application provides an image processing model optimization method, the method comprising:

[0006] Get user image;

[0007] Performing desensitization processing on the user image to obtain first desensitized data corresponding to the user image;

[0008] Extracting marking data corresponding to the first desensitized data from the user image, wherein the marking data is second desensitized data including marking information;

[0009] Based on the first desensitized data and the marked data, the current image processing model is optimized to obtain an optimized image processing model.

[0010] Optionally, before performing desensitization processing on the user image to obtain first desensitized data corresponding to the user image, the method further includes:

[0011] Inputting the user image into the current image processing model to obtain an image processing result output by the current image processing model;

[0012] When the image processing result indicates that the image processing result does not meet the preset conditions, the steps of performing desensitization processing on the user image to obtain first desensitized data corresponding to the user image, and extracting marking data corresponding to the desensitized data from the user image are performed.

[0013] Optionally, a first image size of the first desensitized data is different from a second image size of the user image, and optimizing the current image processing model based on the desensitized data and the labeled data includes:

[0014] Based on a preset adaptation network, adjusting the first image size of the first desensitized data to the second image size;

[0015] The current image processing model is optimized based on the desensitized data of the second image size and the marking data.

[0016] Optionally, extracting the labeled data corresponding to the desensitized data from the user image includes:

[0017] Edge information is extracted from the user image to obtain an edge image, and the edge image is used as the marking data.

[0018] Optionally, the current image processing model can be obtained by following the steps:

[0019] Acquire an initial training data set, wherein the initial training data set includes an open source image subset, a test image subset, and label information corresponding to each open source image and each test image;

[0020] Training the initial image processing model based on the initial training data set to obtain a trained initial image processing model;

[0021] A historical user image set collected within a historical period is obtained, and based on the historical user image set, the trained initial image processing model is optimized to obtain the current image processing model.

[0022] Optionally, performing desensitization processing on the user image to obtain first desensitized data corresponding to the user image includes:

[0023] Performing time-frequency conversion on the user image to obtain frequency domain data of the user image;

[0024] The frequency domain data is processed based on a preset normalization strategy to obtain the first desensitized data.

[0025] In a second aspect, the present application further provides an image processing model optimization method, applied to a terminal, the method comprising:

[0026] Get user image;

[0027] Performing desensitization processing on the user image to obtain first desensitized data corresponding to the user image;

[0028] extracting, from the user image, tagging data corresponding to the first de-identification data, wherein the tagging data is second de-identification data containing tagging information;

[0029] sending the first de-identification data and the tagging data to the cloud, so that the cloud optimizes a current image processing model based on the first de-identification data and the tagging data.

[0030] In a third aspect, the present application also provides an image processing model optimization method, applied to a cloud, and the method comprises:

[0031] receiving first de-identification data and tagging data corresponding to a user image sent by a terminal, wherein the first de-identification data is data obtained by de-identifying the user image, and the tagging data is second de-identification data containing tagging information extracted from the user image;

[0032] optimizing a current image processing model based on the first de-identification data and the tagging data, to obtain an optimized image processing model.

[0033] In a fourth aspect, the present application also provides an image processing model optimization device, comprising:

[0034] a first obtaining module, configured to obtain a user image;

[0035] a first processing module, configured to de-identify the user image to obtain first de-identification data corresponding to the user image;

[0036] a second processing module, configured to extract, from the user image, tagging data corresponding to the first de-identification data, wherein the tagging data is second de-identification data containing tagging information;

[0037] a first optimization module, configured to optimize a current image processing model based on the first de-identification data and the tagging data, to obtain an optimized image processing model.

[0038] In a fifth aspect, the present application also provides an image processing model optimization device, applied to a terminal, and the device comprises:

[0039] a second obtaining module, configured to obtain a user image;

[0040] a third processing module, configured to de-identify the user image to obtain first de-identification data corresponding to the user image;

[0041] a fourth processing module, configured to extract, from the user image, tagging data corresponding to the first de-identification data, wherein the tagging data is second de-identification data containing tagging information;

[0042] A sending module is used to send the first anonymized data and the marked data to the cloud, so that the cloud optimizes the current image processing model based on the first desensitized data and the marked data.

[0043] In a sixth aspect, the present application further provides an image processing model optimization device, which is applied in the cloud, and the device comprises:

[0044] a receiving module, configured to receive first desensitized data and marking data corresponding to a user image sent by a terminal, wherein the first desensitized data is data obtained by desensitizing the user image, and the marking data is second desensitized data extracted from the user image and containing marking information;

[0045] The second optimization module is used to optimize the current image processing model based on the first desensitized data and the marking data to obtain an optimized image processing model.

[0046] In the seventh aspect, an embodiment of the present invention provides an image processing model optimization device, comprising a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors to execute the operating instructions corresponding to the image processing model optimization method provided in the first aspect.

[0047] In an eighth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps corresponding to the image processing model optimization method provided in the first aspect.

[0048] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0049] The solution in the embodiment of the present application, in the process of optimizing the image processing model, obtains a user image, on the one hand desensitizes the user image to obtain first desensitized data corresponding to the user image, and on the other hand extracts marking data corresponding to the desensitized data from the user image, wherein the marking data is second desensitized data containing marking information; based on the first desensitized data and the marking data, the current image processing model is optimized to obtain an optimized image processing model. In the above solution, by extracting the marking data from the user image and marking the first desensitized data, the first desensitized data obtained from the user image can be used to optimize the image processing model. Since the user image is an image collected by the user during the actual use of the image processing model, the image processing model is optimized based on the first desensitized data so that the image processing model is better suitable for the current application scenario, thereby improving the accuracy of the image processing model. BRIEF DESCRIPTION OF DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0051] Figure 1 A flowchart of an image processing model optimization method provided by an embodiment of the present application;

[0052] Figure 2 A system flowchart of an image processing model optimization provided by an embodiment of the present application;

[0053] Figure 3 A flowchart of an image processing model optimization method applied to a terminal provided by an embodiment of the present application;

[0054] Figure 4 A flowchart of an image processing model optimization method applied to a cloud provided by an embodiment of the present application;

[0055] Figure 5 A schematic diagram of an image processing model optimization device provided by an embodiment of the present application;

[0056] Figure 6 A schematic diagram of an image processing model optimization device applied to a terminal provided by an embodiment of the present application;

[0057] Figure 7 A schematic diagram of an image processing model optimization device applied to a cloud provided by an embodiment of the present application;

[0058] Figure 8 A schematic diagram of another image processing model optimization device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0059] The embodiments of the present application provide an image processing model optimization method, device and storage medium, to provide an effective scheme for optimizing the model processing model.

[0060] The technical scheme of the embodiments of the present application is as follows: obtaining a user image; performing desensitization processing on the user image to obtain first desensitization data corresponding to the user image; extracting marked data corresponding to the desensitization data from the user image, wherein the marked data is second desensitization data containing marked information; based on the first desensitization data and the marked data, optimizing a current image processing model to obtain an optimized image processing model.

[0061] The solution in the present application extracts marking data from the user image and marks the first desensitized data, so that the first desensitized data obtained from the user image can be used to optimize the image processing model. Since the user image is an image collected by the user during the actual use of the image processing model, the image processing model is optimized based on the first desensitized data, making the image processing model better suitable for the current application scenario, thereby improving the accuracy of the image processing model.

[0062] In order to better understand the above technical solution, the above technical solution will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0063] First, the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. Furthermore, the character " / " in this document generally indicates an "or" relationship between the associated objects.

[0064] The embodiment of this specification provides an image processing model optimization method, such as Figure 1 As shown, the method includes the following steps:

[0065] Step S101: Obtaining a user image;

[0066] Step S102: performing desensitization processing on the user image to obtain first desensitized data corresponding to the user image;

[0067] Step S103: extracting marking data corresponding to the first desensitized data from the user image, wherein the marking data is second desensitized data containing marking information;

[0068] Step S104: Based on the first desensitized data and the marked data, the current image processing model is optimized to obtain an optimized image processing model.

[0069] The solutions in the embodiments of this specification can be applied to terminal devices deployed with image processing models, to cloud servers deployed with image processing models, or to systems consisting of terminal devices and cloud servers, without limitation here.

[0070] For example, the solution in the embodiments of this specification can be applied to an in-vehicle visual assistance system. The in-vehicle visual assistance system is equipped with an image processing model, which can capture and recognize the preset actions and gestures of people in the car through the image processing model to realize the corresponding functions, thereby improving the driving or riding experience of people in the car.

[0071] For ease of explanation, the image processing model optimization method provided in the embodiments of this specification is described below using the image processing model in the above-mentioned vehicle-mounted visual assistance system as an example.

[0072] In step S101, an image acquisition device may be installed in the vehicle, and the image acquisition device may be used to acquire images of users in the vehicle in real time, wherein the user images may include the user's face, gestures, and other interactive information for human-computer interaction.

[0073] In step 102, after obtaining the user image, the user image is desensitized to remove sensitive information from the user image in order to use it as optimization data for the image processing model, thereby obtaining first desensitized data. It should be noted that the desensitized data cannot be restored or identified by any means, ensuring that user information is not leaked.

[0074] In the embodiments of this specification, user images can be desensitized in a variety of ways, which are not limited here. For example, image desensitization can be achieved by normalizing the pixel values ​​of pixels in the user image, or by removing or deforming sensitive areas of the user image.

[0075] In one embodiment, the first desensitized data can be obtained by: performing time-frequency conversion on the user image to obtain frequency domain data of the user image; and processing the frequency domain data based on a preset normalization strategy to obtain the first desensitized data.

[0076] Specifically, the time-frequency transformation of the user image can be achieved in various ways, such as performing a Fourier transform on the user image to obtain frequency domain data of the user image. After obtaining the frequency domain data, the frequency domain data can be directly normalized to obtain the first desensitized data. Alternatively, the frequency domain data can be processed before normalization and then normalized to obtain the first desensitized data.

[0077] The processing of frequency domain data may include the following steps: removing high-frequency information from the frequency domain data, disrupting the channel order, and fusing information from adjacent frequency domains. After the above processing, normalization is performed according to a preset normalization strategy.

[0078] The normalization strategy can be selected according to actual needs. For example, the preset normalization strategy can be Z-score normalization, Min-Max normalization, etc.

[0079] In one embodiment, the size of the user image is 256*256*1. After desensitization processing, first desensitized data with a size of 32*32*35 is obtained, wherein the first desensitized data is an image that cannot be visualized, and the original user image cannot be restored through the first desensitized data.

[0080] Since the first desensitized data cannot be visualized, in order to enable the first desensitized data to be used for optimization of the image processing model, labeled data corresponding to the desensitized data can be extracted from the user image through step S103.

[0081] It should be noted that to prevent the leakage of sensitive user information, the labeled data is a second desensitized data containing labeled information. The labeled information can be determined based on the task of the image processing model. For example, if the task is to perform gesture recognition on a user image, the labeled information can be hand feature information in the user image. If the task is to perform expression recognition on a user image, the labeled information can be facial feature information in the user image.

[0082] Extracting the marking data corresponding to the desensitized data from the user image can be achieved in a variety of ways. For example, in order to achieve desensitization and retain the marking information, you can first determine whether the area corresponding to the marking information is an area related to the user's sensitive information. For example, determine whether the area corresponding to the marking information is the user's facial area. If not, you can directly remove and cover the area related to the sensitive information and retain other areas. If the area corresponding to the marking information is consistent with the area related to the sensitive information, the sensitive information can be deformed. For example, still taking the area where the sensitive information is located as the facial area as an example, if the area corresponding to the marking information is also the facial area, the facial area can be transformed into a cartoon style, while retaining the facial features, making the user's identity unrecognizable, thereby protecting user information.

[0083] In the embodiment of this specification, step S103 can be implemented in the following manner: extracting edge information from the user image to obtain an edge image, and using the edge image as the marking data.

[0084] Specifically, the method for extracting edge information can be selected based on actual needs. For example, methods such as Canny, Roberts, Sobel, or Prewitt can be used to extract edge information, resulting in a binary image containing the user's outline, namely an edge image. Since the edge image only contains the user's outline information, the user's identity cannot be identified from the edge image and can be regarded as a desensitized image. At the same time, the outline information in the edge image can be used for visual labeling operations, so the edge image can be used as labeling data.

[0085] Further, in step S104, the current image processing model is optimized by the first desensitization data and the labeling data, to obtain an optimized image processing model. Since the first desensitization data is obtained based on the actual application scenario of the image processing model, the optimized model is more suitable for the current application scenario by adjusting and optimizing the first desensitization data and the corresponding labeling data, which is conducive to improving the accuracy of the model.

[0086] Optionally, in the embodiments of the present specification, in order to optimize the current image processing model in a targeted manner, the user images used for model optimization can be screened, that is, not all collected user images are desensitized and used for model optimization, but user images with errors in the output results of the current image processing model are selected to optimize the model, which is conducive to improving the processing accuracy of such user images.

[0087] In the specific implementation process, the following method can be used: inputting the user image into the current image processing model to obtain an image processing result output by the current image processing model; when the image processing result indicates that the image processing result does not meet the preset condition, performing the desensitization processing on the user image to obtain first desensitization data corresponding to the user image, and extracting labeling data corresponding to the desensitization data from the user image.

[0088] Specifically, the preset condition is used to represent that the image processing result is within a correct range, and the preset condition can be set according to actual needs. For example, since image acquisition is performed in real time, the image processing results of a preset number of adjacent user images can be used to determine the preset condition: input a preset number of user images sequentially adjacent to the user image into the image processing model to obtain the image processing result of each adjacent user image, compare the image processing results of the user image and the adjacent user images, if the difference degree of the image processing results meets a preset difference degree, it indicates that the image processing result meets the preset condition, if the difference degree does not meet the preset difference degree, it indicates that the image processing result does not meet the preset condition.

[0089] Taking the task of object detection as an example, when a user image is captured, the user image is input into the image processing model for object detection, and the target detection result is output, such as marking the target with a marker box. Simultaneously, the target detection results corresponding to other frames adjacent to the user image are obtained, and the target detection results of the user image and the adjacent frames are compared. That is, the position of the marker box annotations in each frame is detected. If the error between the annotated position of the marker box in the user image and the annotated position of the marker box in the adjacent frame user image is within a preset error range, the image processing result of the user image meets the preset conditions, and the user image does not need to be desensitized. Conversely, if the error between the annotated position of the marker box in the user image and the annotated position of the marker box in the adjacent frame user image exceeds the preset error range, it indicates that the processing result of the image processing model has deviated, and the user image needs to be desensitized and the labeling information extracted to optimize the image processing model. It should be noted that when optimizing the image processing model, a first desensitized data set and a corresponding labeled data set in which image processing deviations occur can be obtained within a preset period. The preset period can be set according to actual needs, such as 10 minutes, 1 hour, or one week. The current image processing model is optimized using the first desensitized data set and the corresponding labeled data set obtained within the preset period.

[0090] Of course, the preset conditions can also be determined in other ways, for example, based on user feedback. For example, if after obtaining the image processing result of the user image, if user feedback indicating an error in the processing is received, the image processing result is determined to not meet the preset conditions. Those skilled in the art can set these according to actual needs, and a detailed description is not given here.

[0091] Next, the process of obtaining the current image processing model is described.

[0092] In the embodiments of this specification, the current image processing model is the model currently used for image processing, which is obtained through continuous historical optimization. Specifically, the current image processing model can be obtained by: obtaining an initial training data set, wherein the initial training data set includes an open source image subset, a test image subset, and label information corresponding to each open source image and each test image; training the initial image processing model based on the initial training data set to obtain a trained initial image processing model; obtaining a historical user image set collected within a historical period, and optimizing the trained initial image processing model based on the historical user image set to obtain the current image processing model.

[0093] Specifically, the open-source image subset in the initial training dataset can be a collection of images obtained from a public database, and the test image subset can be images collected during the execution of a test task set based on the image processing model. Because the initial training dataset is used to train the initial image processing model, which has not yet been put into practical use, it does not include images actually used by users.

[0094] The initial image processing model can be selected based on actual needs. For example, the initial image processing model can be a CNN (Convolutional Neural Networks) model, an R-CNN (Region CNN) model, etc., without limitation herein. The initial image processing model is iteratively trained using the initial training data set, and the model parameters are gradually adjusted until the number of iterations reaches a set number or the accuracy of the model output reaches a set accuracy. This completes the training process of the initial image processing model and results in a trained image processing model.

[0095] Furthermore, the trained initial image processing model is deployed, that is, put into use, and the user images actually collected are collected during use. It should be noted that the historical period can be any one or more periods between the moment when the trained initial image processing model is put into use and the current moment. In fact, the current image processing model is also obtained through continuous optimization of multiple historical periods. In the process of model optimization, it is also possible to filter out images whose image processing results do not meet the preset conditions in the historical period, and perform desensitization processing and labeling information extraction, and perform model optimization based on these data. Of course, all images collected in the historical period can also be desensitized and labeling information extracted to achieve model optimization, which is not limited here.

[0096] In addition, in the embodiments of this specification, it is considered that the first image size of the first desensitized data may be different from the second image size of the user image. For example, still taking the example mentioned above, the original second image size of the user image is 256*256*1, and the first image size of the first desensitized data is 32*32*35. Since the size of the input data of the image processing model is fixed, that is, the second image size, in order to enable the first desensitized data to be used for model optimization, in the embodiments of this specification, the first image size of the first desensitized data can be adjusted to the second image size through a preset adaptation network, and the current image processing model can be optimized based on the desensitized data of the second image size and the corresponding marking data.

[0097] The function of the adaptation network is to resize the first desensitized data into a size suitable for the image processing model. The way the adaptation network performs resize can be set according to actual needs.

[0098] In the embodiment of this specification, after the optimized image processing model is obtained, the current image processing model is replaced with the optimized image processing model to update the image processing model.

[0099] In order to better understand the image processing model optimization method provided in the embodiments of this specification, the following description will be given by taking the image processing model deployed in a car terminal and the image processing model optimization implemented in the cloud as an example.

[0100] Please refer to Figure 2 In the cloud, the image processing model is first obtained through model training using the test data set and the open source data set. At this time, the image processing model is the initial image processing model, and the initial image processing model is deployed on the terminal, such as in the vehicle-mounted visual assistance system.

[0101] When the terminal is running the in-vehicle visual assistance system, it collects user images in real time. The collected user images are continuously processed by the image processing model deployed on the terminal to obtain image processing results. When there is a deviation in the image processing result, data reflow is triggered. Specifically, the terminal will desensitize the deviated user image on the one hand to obtain the first desensitized data, and on the other hand, extract edge information to obtain edge data for marking, and reflow the first desensitized data and edge data to the cloud.

[0102] Furthermore, after receiving the first desensitized data and edge data, the cloud aggregates them into a user data set, and resizes the desensitized data in the user data set through an adaptation network, performs model training based on the corresponding edge data, obtains an optimized image processing model, and deploys the optimized image processing model to the terminal.

[0103] The above steps are executed repeatedly to continuously optimize and update the image processing model on the end side, making the image processing model applicable to various scenarios and improving the accuracy and generalization of the model.

[0104] Based on the same inventive concept, the embodiment of this specification also provides an image processing model optimization method, which is applied to a terminal, such as Figure 3 As shown, the method includes the following steps:

[0105] Step S301: Obtain user image;

[0106] Step S302: performing desensitization processing on the user image to obtain first desensitized data corresponding to the user image;

[0107] Step S303: extracting marking data corresponding to the first desensitized data from the user image, wherein the marking data is second desensitized data containing marking information;

[0108] Step S304: Send the first desensitized data and the marked data to the cloud, so that the cloud optimizes the current image processing model based on the first desensitized data and the marked data.

[0109] The specific implementation steps of the above method have been described in detail in the previous description and will not be elaborated here.

[0110] Based on the same inventive concept, an embodiment of this specification further provides an image processing model optimization method, which is applied to the cloud. The method includes the following steps:

[0111] Step S401: receiving first desensitized data and marking data corresponding to a user image sent by a terminal;

[0112] The first desensitized data is data obtained by desensitizing the user image, and the marked data is second desensitized data extracted from the user image and containing marking information;

[0113] Step S402: Based on the first desensitized data and the marked data, the current image processing model is optimized to obtain an optimized image processing model.

[0114] The specific implementation steps of the above method have been described in detail in the previous description and will not be elaborated here.

[0115] Based on the same inventive concept, the embodiment of this specification also provides an image processing model optimization device, such as Figure 5 As shown, the device includes:

[0116] A first acquisition module 501 is used to acquire a user image;

[0117] A first processing module 502 is configured to perform desensitization processing on the user image to obtain first desensitized data corresponding to the user image;

[0118] A second processing module 503 is configured to extract marking data corresponding to the desensitized data from the user image, wherein the marking data is second desensitized data including marking information;

[0119] The first optimization module 504 is used to optimize the current image processing model based on the first desensitized data and the marked data to obtain an optimized image processing model.

[0120] Optionally, the device further comprises:

[0121] An image processing module, configured to input the user image into the current image processing model and obtain an image processing result output by the current image processing model;

[0122] An execution module is used to perform the desensitization processing on the user image when the image processing result indicates that the image processing result does not meet the preset conditions to obtain the first desensitized data corresponding to the user image, and the step of extracting the marking data corresponding to the desensitized data from the user image.

[0123] Optionally, the first image size of the first desensitized data is different from the second image size of the user image, and the first optimization module 504 is configured to:

[0124] Based on a preset adaptation network, adjusting the first image size of the first desensitized data to the second image size;

[0125] The current image processing model is optimized based on the desensitized data of the second image size and the marking data.

[0126] Optionally, the second processing module 503 is configured to:

[0127] Edge information is extracted from the user image to obtain an edge image, and the edge image is used as the marking data.

[0128] Optionally, the current image processing model can be obtained by following the steps:

[0129] Acquire an initial training data set, wherein the initial training data set includes an open source image subset, a test image subset, and label information corresponding to each open source image and each test image;

[0130] Training the initial image processing model based on the initial training data set to obtain a trained initial image processing model;

[0131] A historical user image set collected within a historical period is obtained, and based on the historical user image set, the trained initial image processing model is optimized to obtain the current image processing model.

[0132] Optionally, the first processing module 502 is configured to:

[0133] Performing time-frequency conversion on the user image to obtain frequency domain data of the user image;

[0134] The frequency domain data is processed based on a preset normalization strategy to obtain the first desensitized data.

[0135] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0136] Based on the same inventive concept, the embodiment of this specification also provides an image processing model optimization device, which is applied to a terminal, such as Figure 6 As shown, the device includes:

[0137] The second acquisition module 601 is used to acquire a user image;

[0138] A third processing module 602 is configured to perform desensitization processing on the user image to obtain first desensitized data corresponding to the user image;

[0139] A fourth processing module 603 is configured to extract, from the user image, marking data corresponding to the first desensitized data, wherein the marking data is second desensitized data including marking information;

[0140] The sending module 604 is used to send the first entrusted data and the marked data to the cloud, so that the cloud optimizes the current image processing model based on the first desensitized data and the marked data.

[0141] Specifically, the terminal can be a mobile terminal, a car terminal, etc. An image processing model is deployed on the terminal, which can process the user images collected by the terminal.

[0142] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0143] Based on the same inventive concept, the embodiment of this specification also provides an image processing model optimization device, such as Figure 7 As shown, applied to the cloud, the device includes:

[0144] A receiving module 701 is configured to receive first desensitized data and marking data corresponding to a user image sent by a terminal, wherein the first desensitized data is data obtained by desensitizing the user image, and the marking data is second desensitized data extracted from the user image and containing marking information;

[0145] The second optimization module 702 is used to optimize the current image processing model based on the first desensitized data and the marked data to obtain an optimized image processing model.

[0146] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0147] Based on the same inventive concept, the embodiment of this specification also provides an image processing model optimization device, such as Figure 8 As shown, it includes a memory 808, a processor 802, and a computer program stored in the memory 808 and executable on the processor 802. When the processor 802 executes the program, the steps of any one of the above-mentioned image processing model optimization methods are implemented.

[0148] Among them, Figure 8 In the embodiment of the present invention, a bus architecture (represented by bus 800) is shown. Bus 800 may include any number of interconnected buses and bridges, and bus 800 links together various circuits including one or more processors represented by processor 802 and memory represented by memory 808. Bus 800 may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. Bus interface 806 provides an interface between bus 800 and receiver 801 and transmitter 803. Receiver 801 and transmitter 803 may be the same component, namely a transceiver, which provides a unit for communicating with various other devices over a transmission medium. Processor 802 is responsible for managing bus 800 and general processing, while memory 808 may be used to store data used by processor 802 when performing operations.

[0149] In addition, an embodiment of the present invention also provides a front end, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, it implements the steps of any one of the image processing model optimization methods described in the second aspect above.

[0150] Correspondingly, an embodiment of the present invention also provides a cloud, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, it implements the steps of any one of the image processing model optimization methods described in the third aspect above.

[0151] Based on the same inventive concept, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-mentioned image processing model optimization methods.

[0152] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that performs the functions specified in one or more boxes.

[0153] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0154] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0155] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0156] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A method for optimizing an image processing model, characterized in that: The method comprises: Get user image; Performing desensitization processing on the user image to obtain first desensitized data corresponding to the user image; Extracting marking data corresponding to the first desensitized data from the user image, including: extracting edge information from the user image to obtain an edge image, and using the edge image as the marking data, wherein the marking data is second desensitized data including the marking information; Based on the first desensitized data and the marked data, the current image processing model is optimized to obtain an optimized image processing model.

2. The method according to claim 1, wherein Before performing desensitization processing on the user image to obtain first desensitized data corresponding to the user image, the method further includes: Inputting the user image into the current image processing model to obtain an image processing result output by the current image processing model; When the image processing result indicates that the image processing result does not meet the preset conditions, the steps of performing desensitization processing on the user image to obtain first desensitized data corresponding to the user image, and extracting marking data corresponding to the desensitized data from the user image are performed.

3. The method according to claim 1, wherein A first image size of the first desensitized data is different from a second image size of the user image, and optimizing the current image processing model based on the first desensitized data and the labeled data includes: Based on a preset adaptation network, adjusting the first image size of the first desensitized data to the second image size; The current image processing model is optimized based on the desensitized data of the second image size and the marking data.

4. The method according to claim 1, wherein The current image processing model can be obtained by the following steps: Acquire an initial training data set, wherein the initial training data set includes an open source image subset, a test image subset, and label information corresponding to each open source image and each test image; Training the initial image processing model based on the initial training data set to obtain a trained initial image processing model; A historical user image set collected within a historical period is obtained, and based on the historical user image set, the trained initial image processing model is optimized to obtain the current image processing model.

5. The method according to claim 1, wherein The performing desensitization processing on the user image to obtain first desensitized data corresponding to the user image includes: Performing time-frequency conversion on the user image to obtain frequency domain data of the user image; The frequency domain data is processed based on a preset normalization strategy to obtain the first desensitized data.

6. A method for optimizing an image processing model, characterized in that: Applied to a terminal, the method includes: Get user image; Performing desensitization processing on the user image to obtain first desensitized data corresponding to the user image; Extracting marking data corresponding to the first desensitized data from the user image, including: extracting edge information from the user image to obtain an edge image, and using the edge image as the marking data, wherein the marking data is second desensitized data including the marking information; The first desensitized data and the marked data are sent to the cloud, so that the cloud optimizes the current image processing model based on the first desensitized data and the marked data.

7. A method for optimizing an image processing model, characterized in that: Applied to the cloud, the method includes: Receiving first desensitized data and labeled data corresponding to the user image sent by the terminal, wherein the first desensitized data is data obtained by desensitizing the user image, and the labeled data is second desensitized data extracted from the user image and containing labeled information; Optimizing the current image processing model based on the first desensitized data and the marked data to obtain an optimized image processing model; The marking data is obtained by performing edge information extraction on the user image to obtain an edge image, and using the edge image as the marking data.

8. An image processing model optimization device, characterized in that: include: A first acquisition module is used to acquire a user image; A first processing module, configured to perform desensitization processing on the user image to obtain first desensitized data corresponding to the user image; a second processing module, configured to extract marking data corresponding to the desensitized data from the user image, comprising: extracting edge information from the user image to obtain an edge image, and using the edge image as the marking data, wherein the marking data is second desensitized data including the marking information; The first optimization module is used to optimize the current image processing model based on the first desensitized data and the marking data to obtain an optimized image processing model.

9. An image processing model optimization device, characterized in that: Applied to a terminal, the device includes: A second acquisition module is used to acquire a user image; a third processing module, configured to perform desensitization processing on the user image to obtain first desensitized data corresponding to the user image; a fourth processing module, configured to extract marking data corresponding to the first desensitized data from the user image, comprising: extracting edge information from the user image to obtain an edge image, and using the edge image as the marking data, wherein the marking data is the second desensitized data including the marking information; A sending module is used to send the first desensitized data and the marked data to the cloud, so that the cloud optimizes the current image processing model based on the first desensitized data and the marked data.

10. An image processing model optimization device, characterized in that: Applied to the cloud, the device includes: a receiving module, configured to receive first desensitized data and marking data corresponding to a user image sent by a terminal, wherein the first desensitized data is data obtained by desensitizing the user image, and the marking data is second desensitized data extracted from the user image and containing marking information; A second optimization module is used to optimize the current image processing model based on the first desensitized data and the marked data to obtain an optimized image processing model; The marking data is obtained by performing edge information extraction on the user image to obtain an edge image, and using the edge image as the marking data.

11. An image processing model optimization device, characterized in that: The invention comprises a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors to execute the operating instructions corresponding to the method according to any one of claims 1 to 7 contained in the one or more programs.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method steps according to any one of claims 1 to 7 are implemented.

Images