A model processing method and apparatus, a storage medium, and an electronic device

Abstract

This specification discloses a model processing method, apparatus, storage medium, and electronic device. The method includes: deploying an initial image processing model to at least one client in a target transaction scenario; obtaining at least one image processing sample pair for the initial image processing model from the client, wherein the image processing sample pair includes input image data and model output data; performing model calibration processing on the initial image processing model based on the at least one image processing sample pair to generate an image processing model; and then performing transaction processing on the target transaction scenario based on the image processing model.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a model processing method, apparatus, storage medium and electronic device. Background Technology

[0002] With the rapid development of computer technology, the application of image processing models to process images in real-world scenarios such as image recognition, image classification, image segmentation, and image privacy protection is becoming increasingly widespread. Before using an image processing model, it is necessary to train the model first.

[0003] During the training process of the image processing model, training data is collected, and the image processing model is trained offline based on the training data. After the training is completed, the trained image processing model is deployed to real-world business scenarios for image processing. Summary of the Invention

[0004] This specification provides a model processing method, apparatus, storage medium, and electronic device, the technical solution of which is as follows:

[0005] Firstly, this specification provides a model processing method applied to a service platform, the method comprising:

[0006] An initial image processing model is deployed to at least one client in the target transaction scenario, and at least one image processing sample pair for the initial image processing model is obtained from the client, the image processing sample pair including input image data and model output data;

[0007] Based on the at least one image processing sample pair, the initial image processing model is calibrated to generate an image processing model;

[0008] The target transaction scenario is processed based on the image processing model.

[0009] Secondly, this specification provides a model processing method applied to a client, the method comprising:

[0010] Obtain an initial image processing model from the service platform and deploy the initial image processing model to the target transaction scenario;

[0011] Based on the initial image processing model, the input image data is processed to obtain the model output data;

[0012] Image processing sample pairs are generated based on the input image data and model output data. The image processing sample pairs are sent to the service platform so that the service platform can perform model calibration processing on the initial image processing model based on at least one of the image processing sample pairs to generate an image processing model, and perform transaction processing on the target transaction scenario based on the image processing model.

[0013] Thirdly, this specification provides a model processing apparatus, the apparatus comprising:

[0014] The sample acquisition module is used to deploy an initial image processing model to at least one client in the target transaction scenario, and to acquire at least one image processing sample pair for the initial image processing model from the client. The image processing sample pair includes input image data and model output data.

[0015] The model generation module is used to perform model calibration processing on the initial image processing model based on the at least one image processing sample pair to generate an image processing model.

[0016] The model update module is used to perform transaction processing on the target transaction scenario based on the image processing model.

[0017] Fourthly, this specification provides a model processing apparatus, the apparatus comprising:

[0018] The model deployment module is used to obtain an initial image processing model from the service platform and deploy the initial image processing model to the target transaction scenario;

[0019] The model processing module is used to process the input image data based on the initial image processing model to obtain model output data;

[0020] The sample sending module is used to generate image processing sample pairs based on the input image data and model output data, and send the image processing sample pairs to the service platform, so that the service platform can perform model calibration processing on the initial image processing model based on at least one of the image processing sample pairs to generate an image processing model, and perform transaction processing on the target transaction scenario based on the image processing model.

[0021] Fifthly, this specification provides a computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the above-described method steps.

[0022] Sixthly, this specification provides a computer program product that stores a plurality of instructions adapted for loading by a processor and executing the above-described method steps.

[0023] In a seventh aspect, this specification provides an electronic device that may include: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the above-described method steps.

[0024] The beneficial effects of the technical solutions provided in some embodiments of this specification include at least the following:

[0025] In one or more embodiments of this specification, the service platform deploys an initial image processing model to at least one client in the target transaction scenario for model trial operation. The client processes the input image data in the target transaction scenario based on the initial image processing model to obtain model output data. Based on this, the service platform can collect at least one image processing sample pair from the client for the initial image processing model, and then use the image processing sample pair to perform model calibration on the initial image processing model to generate an image processing model. Then, the service platform can perform transaction processing on the target transaction scenario based on this image processing model. The calibrated image processing model can resist the distribution differences between training data and actual online data, has good model robustness, optimizes the model processing process, and effectively enhances the stability and scenario generalization ability of the model after being deployed to the target transaction scenario. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in this specification or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a scene diagram of a model processing system provided in this manual;

[0028] Figure 2 This is a flowchart illustrating another model processing method provided in this manual;

[0029] Figure 3 This is a flowchart illustrating another model processing method provided in this manual;

[0030] Figure 4 This is a flowchart illustrating another model processing method provided in this manual;

[0031] Figure 5 This is a flowchart illustrating another model processing method provided in this manual;

[0032] Figure 6 This is a schematic diagram of the structure of a model processing device provided in this specification;

[0033] Figure 7 This is a schematic diagram of the structure of a sample acquisition module provided in this specification;

[0034] Figure 8 This is a schematic diagram of the structure of a model generation module provided in this manual;

[0035] Figure 9 This is a structural diagram of a structural adjustment unit provided in this specification;

[0036] Figure 10 This is a schematic diagram of another model processing device provided in this specification;

[0037] Figure 11 This is a schematic diagram of the structure of an electronic device provided in this specification;

[0038] Figure 12 This is a schematic diagram of another electronic device provided in this manual;

[0039] Figure 13 This is a schematic diagram of the operating system and user space provided in this manual;

[0040] Figure 14 yes Figure 13 Architecture diagram of the Android operating system in China;

[0041] Figure 15 yes Figure 13 Architecture diagram of the iOS operating system. Detailed Implementation

[0042] The technical solutions in this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0043] In the description of this application, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. In the description of this application, it should be noted that, unless otherwise expressly specified and limited, "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances. Furthermore, in the description of this application, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist; for example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0044] The present application will now be described in detail with reference to specific embodiments.

[0045] Please see Figure 1 This is a scene diagram of a model processing system provided in this specification. Figure 1 As shown, the model processing system may include at least a client cluster and a service platform 100.

[0046] The client cluster may include at least one client, such as Figure 1 As shown, it specifically includes client 1 corresponding to user 1, client 2 corresponding to user 2, ..., client n corresponding to user n, where n is an integer greater than 0.

[0047] Each client in a client cluster can be an electronic device with communication capabilities, including but not limited to: wearable devices, handheld devices, personal computers, tablets, in-vehicle devices, smartphones, computing devices, or other processing devices connected to a wireless modem. Electronic devices may have different names in different networks, such as: user equipment, access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent or user device, cellular phone, cordless phone, personal digital assistant (PDA), and electronic devices in 5G networks or future evolved networks.

[0048] The service platform 100 can be a standalone server device, such as a rack-mount, blade, tower, or cabinet-type server device, or a workstation, mainframe, or other hardware device with strong computing power; or it can be a server cluster composed of multiple servers. The servers in the service cluster can be composed in a symmetrical manner, wherein each server is functionally and hierarchically equivalent in the transaction chain, and each server can provide services independently. The independent provision of services can be understood as not requiring the assistance of other servers.

[0049] In one or more embodiments of this specification, the service platform 100 can establish a communication connection with at least one client in the client cluster, and complete data interaction during model processing based on the communication connection, such as online transaction data interaction. For example, the service platform 100 can transmit the model data of the initial image processing model to at least one client based on the model processing method of this specification, and the client can deploy the initial image processing model to the target transaction scenario; or the client can perform image processing on the input image data in the target transaction scenario based on the initial image processing model to obtain model output data, and the client can transmit image processing sample pairs containing input image data and model output data to the service platform through the communication connection; or the service platform can transmit the image processing model calibrated based on the image processing sample pairs to several clients, and the clients can officially launch the image processing model in the target transaction scenario, and so on.

[0050] It should be noted that the service platform 100 establishes a communication connection with at least one client in the client cluster via a network for interactive communication. This network can be a wireless network or a wired network. Wireless networks include, but are not limited to, cellular networks, wireless LANs, infrared networks, or Bluetooth networks. Wired networks include, but are not limited to, Ethernet, universal serial bus (USB), or controller area networks. In one or more embodiments of the specification, technologies and / or formats including Hyper Text Markup Language (HTML), Extensible Markup Language (XML), etc., are used to represent data exchanged over the network (such as target compressed packets). Furthermore, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), and Internet Protocol Security (IPsec) can be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.

[0051] The model processing system embodiments provided in this specification and the model processing methods described in one or more embodiments belong to the same concept. The execution entity corresponding to the model processing method involved in one or more embodiments of this specification can be the aforementioned service platform 100; the execution entity corresponding to the model processing method involved in one or more embodiments of this specification can also be the electronic device corresponding to the client, which is determined based on the actual application environment. The implementation process of the model processing system embodiments can be found in the following method embodiments, which will not be repeated here.

[0052] based on Figure 1 The following is a detailed description of the model processing methods provided by one or more embodiments of this specification, as illustrated in the scene diagram.

[0053] Please see Figure 2 This document provides a flowchart illustrating a model processing method according to one or more embodiments of this specification. This method can be implemented using a computer program and can run on a background investigation device based on the von Neumann architecture. The computer program can be integrated into an application or run as a standalone tool application. The model processing device can be a service platform.

[0054] Specifically, the model processing method includes:

[0055] S102: Deploy the initial image processing model to at least one client in the target transaction scenario, and obtain at least one image processing sample pair for the initial image processing model from the client, wherein the image processing sample pair includes input image data and model output data.

[0056] In real-world business scenarios, machine learning-based image processing models are often used to further process image data generated by these scenarios. Image processing models can be applied to different scenarios depending on the machine vision task. In other words, image processing models can be neural networks suitable for various machine vision tasks, such as image recognition, image classification, image segmentation, image detection, pose estimation, video frame analysis, and image privacy protection. Furthermore, initial image processing models trained for different machine vision tasks can be deployed in corresponding target business scenarios. For example, an initial image processing model trained for an image recognition task can be deployed in an image recognition business scenario, an initial image processing model trained for an image privacy protection task can be deployed in an image privacy protection business scenario, an initial image processing model trained for an image classification task can be deployed in an image classification business scenario, and so on.

[0057] Understandably, machine learning often involves differences in the distribution of offline labeled training data and online transaction data. In this application: image sample data from the corresponding transaction scenario is collected in advance as offline image training data and labeled accordingly to obtain labeled training data. This labeled training data is used to train the machine learning-based model. After the offline model training conditions are met, the initial image processing model is obtained.

[0058] In illustrative terms, the initial image processing model can be understood as the trained image processing model that meets the conditions for ending offline model training. In real-world transaction scenarios, deploying this initial image processing model online often reveals significant discrepancies between the offline-trained model and the image data distribution of the online service during the online transaction processing phase. This can lead to severe overfitting of the offline training data, resulting in unstable image processing output and a sharp decline in model performance after formal online deployment, ultimately causing poor performance in everyday transaction scenarios. The model processing methods described in this manual can resolve these issues.

[0059] Furthermore, the initial image processing model can be created by fitting one or more machine learning models such as Convolutional Neural Network (CNN), Deep Neural Network (DNN), Recurrent Neural Networks (RNN), embedding models, Gradient Boosting Decision Tree (GBDT) models, and Logistic Regression (LR) models.

[0060] In a schematic manner, the service platform pre-builds an initial machine learning model offline for the target transaction scenario. The initial machine learning model is set based on image processing tasks such as image recognition, image classification, and image privacy protection to adapt to the target transaction scenario. The initial machine learning model is trained offline using offline image sample data. When the offline model termination condition is met, a trained target image processing model is obtained. Then, the target image processing model is used as the initial image processing model and deployed online.

[0061] Indicatively, the service platform can deploy an initial image processing model to at least one client associated with the target transaction scenario, so that the client processes the input image data in the target transaction scenario based on the initial image processing model to obtain model output data;

[0062] Optionally, the service platform can build an online transaction testing environment containing at least one client. The online transaction testing environment contains a target number (e.g., 100) of clients as model testing devices. After the initial image processing model is calibrated by the online training model, the generated image processing model can be deployed online on a large scale, such as distributing the generated image processing model to all clients associated with the target transaction scenario.

[0063] Furthermore, the client receiving the initial image processing model can apply it to the target transaction scenario for image processing. This target scenario generates input image data, which is the input data for the initial image processing model. The client inputs this input image data into the initial image processing model for processing, and the initial image processing model outputs its output data. In other words, the initial image processing model processes the input image data from the target transaction scenario to obtain the model's output data.

[0064] Understandably, the data type of the model's output data is determined by the machine vision task. For example, if the machine vision task is image recognition, the model output data will be the image recognition result; if the machine vision task is image privacy protection, the model output data will be the anonymized image; if the machine vision task is image classification, the model output data will be the image classification result; if the machine vision task is image segmentation, the model output data will be the image segmentation result, and so on. The specific type of the model's output data is not limited here; it is determined based on the actual transaction.

[0065] Furthermore, the service platform can instruct the client to collect at least one set of input image data and model output data for the initial image processing model; this set of input image data and model output data constitutes an image processing sample pair. The number of image processing sample pairs can be multiple.

[0066] Optionally, the service platform may obtain at least one image processing sample pair from the client.

[0067] Optionally, several clients may actively collect image processing sample pairs and send them to the service platform. In this case, the service platform can receive image processing sample pairs sent by at least one client. These image processing sample pairs include input image data and model output data.

[0068] S104: Based on the at least one image processing sample pair, perform model calibration processing on the initial image processing model to generate an image processing model.

[0069] The purpose of the model calibration process is to enable the initial image processing model to adapt to the online target transaction scenario, possessing a certain degree of robustness. It should exhibit good robustness against changes in online image input data (such as user online transaction image data), thereby improving the model's processing performance after deployment. The performance difference of the initial image processing model in the new target transaction scenario lies in the domain difference between the feature space where the model resides offline and the feature space corresponding to the new online scenario. This domain difference manifests in features such as image quality and image position / pose. Therefore, the model processing methods described in one or more embodiments of this specification can be executed to perform model calibration on the feature space where the model features reside, thereby adapting it to the feature space of the new online target transaction scenario.

[0070] In one feasible implementation, some network layers of the initial image processing model can be re-encoded and trained based on at least one image processing sample, so that the model features (usually feature vectors) extracted by the re-encoded and trained model during the model processing can be adapted to the feature space of the new online target transaction scenario.

[0071] Optionally, the model architecture parameters of at least one network layer of the initial image processing model can be adjusted, and then the initial image processing model can be retrained using at least one of the aforementioned image processing samples until the model ends training conditions are met, thereby generating the image processing model.

[0072] In one feasible implementation, a new model network layer structure can be added to the initial image processing model, and then the initial image processing model can be retrained using at least one of the aforementioned image processing samples. The retraining usually focuses on updating the structural parameters of the newly added model network layer structure, while the structural parameters of the remaining network layers remain unchanged, until the model ends training conditions are met, and an image processing model is generated.

[0073] In one or more embodiments of this specification, the model termination condition may include, for example, the value of the loss function being less than or equal to a preset loss function threshold, or the number of iterations reaching a preset threshold. Specific model termination conditions can be determined based on actual circumstances and will not be elaborated here.

[0074] S106: Perform transaction processing on the target transaction scenario based on the image processing model.

[0075] Understandably, after performing model calibration on the initial image processing model based on the at least one image processing sample pair to generate the image processing model, the image processing model can then be deployed online and adapted to the target transaction scenario.

[0076] As an illustration, the service platform can deploy the image processing model to a large number of clients associated with the target transaction scenario, so that these clients can accurately process the input image data generated online in the target transaction scenario based on the image processing model to obtain model output data with good processing effect;

[0077] Understandably, if a machine vision task is used for image recognition, then the image processing model is used for image recognition. The service platform deploys the image processing model for the image recognition scenario, and then performs image recognition on the image data in the target scenario based on the image processing model, outputting the image recognition result. If a machine vision task is used for image classification, then the image processing model is used for image classification. The service platform deploys the image processing model for the image classification scenario, and then performs image classification on the image data in the target scenario based on the image processing model, outputting the image classification result. If a machine vision task is used for image privacy protection, then the image processing model is used for image privacy protection. The service platform deploys the image processing model for the image privacy protection scenario, and then performs image desensitization on the image data in the target scenario based on the image processing model, outputting the desensitized image, and so on.

[0078] In one or more embodiments of this specification, the service platform deploys an initial image processing model to at least one client in the target transaction scenario for model trial operation. The client processes the input image data in the target transaction scenario based on the initial image processing model to obtain model output data. Based on this, the service platform can collect at least one image processing sample pair from the client for the initial image processing model, and then use the image processing sample pair to perform model calibration on the initial image processing model to generate an image processing model. Then, the service platform can perform transaction processing on the target transaction scenario based on this image processing model. The calibrated image processing model can resist the distribution differences between training data and actual online data, has good model robustness, optimizes the model processing process, and effectively enhances the stability and scenario generalization ability of the model after being deployed to the target transaction scenario.

[0079] Please see Figure 3 , Figure 3 This is a flowchart illustrating another embodiment of a model processing method proposed in one or more embodiments of this specification. Specifically:

[0080] S202: Deploy the initial image processing model to at least one client in the target transaction scenario, and obtain at least one image processing sample pair for the initial image processing model from the client, wherein the image processing sample pair includes input image data and model output data;

[0081] For details, please refer to the method steps in other embodiments of this specification, which will not be repeated here.

[0082] S204: Perform verification and screening on the at least one image processing sample pair to obtain at least one normal image processing sample pair and one abnormal image processing sample pair;

[0083] The verification and screening process can be understood as follows: the service platform verifies the input results of the image processing samples. By judging whether each set of input image data and model output data meets expectations, that is, whether the model output data output by the initial image processing model for the input image data meets expectations, the normal image processing sample pairs that meet expectations and those that do not meet expectations can be distinguished.

[0084] In one feasible implementation, image processing specifications can be set for the initial image processing model. These specifications are used to measure whether the model's output data meets the expectations of the machine vision task. Different machine vision tasks correspond to different image processing specifications. For example, image recognition tasks correspond to image recognition output specifications, which are used to measure and even verify the image recognition effect of the initial image processing model's output data. Similarly, image classification tasks correspond to image classification output specifications, which are used to measure and even verify the image classification effect of the initial image processing model's output data. Furthermore, image classification tasks correspond to image privacy protection tasks, which are used to measure and even verify the image desensitization effect of the initial image processing model's output data, i.e., to determine whether the latent model output data after image desensitization meets privacy protection requirements, and so on.

[0085] Specifically, the service platform can perform verification and screening on at least one image processing sample pair based on the set image processing specifications to determine at least one normal image processing sample pair that matches the image processing specifications, and at least one normal image processing sample pair that does not match the image processing specifications.

[0086] As an illustration, the service platform can call the expert service, which verifies and filters each group of image processing samples based on image processing specifications to detect whether the model output data in the image processing samples meets the image processing specifications, and to determine the normal image processing sample pairs that meet the image processing specifications and the abnormal image processing sample pairs that do not meet the image processing specifications.

[0087] Understandably, by deploying the initial image processing model to at least one client and conducting trial operations under the target transaction scenario, data from the initial image processing model is collected for sample accumulation, i.e., image processing samples. Then, through verification and filtering, at least one pair of normal image processing samples and one pair of abnormal image processing samples can be obtained. Based on these samples, the model structure can be adjusted to meet actual online deployment requirements and improve model robustness and scenario generalization ability.

[0088] S206: Based on the image processing normal sample pairs and the image processing abnormal sample pairs, perform structural adjustment processing on the initial image processing model to obtain a reference image processing model;

[0089] In one feasible implementation, the service platform can evaluate the performance of the initial image processing model in real time based on the distribution of the number of normal and abnormal image processing samples. Based on the performance evaluation, the platform can then structurally adjust the initial image processing model and train the adjusted model using samples. Specifically:

[0090] 1. The service platform can determine the performance evaluation parameters for the initial image processing model based on the normal image processing sample pairs and the abnormal image processing sample pairs.

[0091] The performance evaluation parameters are used to measure the performance of the initial image processing model in the current target transaction scenario. The performance evaluation parameters can quantify the degree of fit between the initial image processing model and the target transaction scenario. The performance evaluation parameters are calculated based on at least the number of normal image processing sample pairs and abnormal image processing sample pairs.

[0092] Optionally, the performance evaluation parameter can be based on the ratio of the number of normal sample pairs to the number of abnormal sample pairs in image processing. The larger the ratio, the worse the model performance.

[0093] Optionally, the performance evaluation parameter can be the ratio of the first number of abnormal image processing sample pairs to the second number of total sample pairs (i.e., the total number of all normal image processing sample pairs and the total number of abnormal image processing sample pairs). The larger the ratio, the worse the model performance.

[0094] Indicatively, the service platform can acquire, in real time or periodically, a first number of abnormal image processing sample pairs and a second number of normal image processing sample pairs; and determine the model running time of the initial image processing model; the model running time is the duration for which the initial image processing model is deployed to the client.

[0095] Indicatively, the service platform then determines the sample accumulation rate based on the first quantity, the second quantity, and the model running time, and uses the sample accumulation rate as a performance evaluation parameter for the initial image processing model.

[0096] The cumulative rate of the samples conforms to the following formula:

[0097] v=(N1 / N2) / t

[0098] Where v is the cumulative rate of the samples, N1 is the first quantity, N2 is the second quantity, and t is the model running time.

[0099] Understandably, the service platform monitors the cumulative rate of the initial image processing model by: counting the number of samples N1 in datasets that do not meet the requirements (such as processing abnormal sample pairs) and the number of samples N2 in datasets that meet the requirements (such as processing normal sample pairs), and the trial operation time t corresponding to the collection of these samples, i.e., the operating time of the initial image processing model on the client side. The cumulative rate of samples provides a direct reflection of the degree to which the initial image processing model fits the new target transaction scenario. A higher cumulative rate indicates that the current initial image processing model performs worse in the new target transaction scenario. In other words, different degrees of structural adjustments are made based on the performance evaluation parameters.

[0100] 2. The service platform determines the model network adjustment layers for the initial image processing model based on performance evaluation parameters;

[0101] Optionally, the specific type of model network adjustment layer in the initial image processing model can be determined based on performance evaluation parameters. For example, the model network adjustment layer can be an input layer, a representation layer, an encoding layer, a decoding layer, etc.

[0102] Optionally, new model network adjustment layers can be determined based on performance evaluation parameters for the initial image processing model. These new model network adjustment layers can be input layers, representation layers, encoding layers, decoding layers, etc.

[0103] Different reference evaluation ranges can be set, each corresponding to a different network structure adjustment strategy. These strategies must include at least the method of adjusting network layers (e.g., adding a new network layer, adjusting an existing network layer), the type of network layer, and the number of network layers. By determining which performance evaluation parameters fall into the target evaluation range, the corresponding network structure adjustment strategy is obtained, and then structural adjustment is performed based on this strategy. For example, if a strategy indicates optimization of the structural parameters of the original input layer, then the indicated structural parameters of the original input layer are adjusted. Similarly, if a strategy indicates adding a new model network adjustment layer (e.g., an encoding layer), then the indicated new model network adjustment layer is connected to the initial image processing model.

[0104] It should be noted that there are no restrictions on the specific model structure adjustment methods here, and they can be set based on the actual application situation.

[0105] In a specific implementation scenario, the service platform can obtain at least one evaluation reference threshold for the performance evaluation parameters, and then determine at least one model network adjustment layer for the initial image processing model based on the at least one evaluation reference threshold and the performance evaluation parameters.

[0106] The number of evaluation reference thresholds can be multiple, and the evaluation reference thresholds are threshold values ​​or critical values ​​set for performance evaluation parameters.

[0107] Schematic, the evaluation reference threshold may include at least a first reference threshold and a second reference threshold, specifically determining at least one model network adjustment layer for the initial image processing model, as follows:

[0108] If the performance evaluation parameter is less than the first reference threshold, then a first number of model network adjustment layers are determined;

[0109] If the performance evaluation parameter is greater than or equal to the first reference threshold and less than or equal to the second reference threshold, then a second number of model network adjustment layers are determined.

[0110] If the performance evaluation parameter is greater than the second reference threshold, then a third number of model network adjustment layers are determined.

[0111] Optionally, the model network adjustment layer can be a model network layer in the initial image processing model, or a model network layer that needs to be added to the original network structure of the initial image processing model. The type of model network layer can be a fit of one or more of the network layer types corresponding to the machine learning models involved in related technologies, such as a fully connected layer (FC layer).

[0112] For example, suppose the first reference threshold is denoted as v1, and the second reference threshold is denoted as v2(v1). <v2)。

[0113] If the performance evaluation parameter - the cumulative sample rate v - is less than v1, then the model network adjustment layer for the initial image processing model is determined to be the first number, such as one.

[0114] If the performance evaluation parameter – the sample accumulation rate v – is between v1 and v2, then the model network adjustment layers for the initial image processing model are determined to be the second number. For example, the second number could be two.

[0115] If the performance evaluation parameter – the cumulative sample rate v – is greater than or equal to v², then the model network adjustment layers for the initial image processing model are determined to be a third number. For example, the third number could be three.

[0116] Understandably, performance evaluation parameters reflect the degree of matching between the current initial image processing model and the new target transaction scenario. It can be understood that the higher the value of the performance evaluation parameter, the lower the degree of matching usually is. Based on this, different numbers of model networks need to be adjusted to improve the matching degree of the current model.

[0117] 3. Based on the model network adjustment layer, the initial image processing model is structurally adjusted to obtain a reference image processing model.

[0118] Understandably, after determining the model network adjustment layers, the initial image processing model is then subjected to structural adjustment processing based on the corresponding structural adjustment processing method.

[0119] Schematic, the initial image processing model can be a model constructed from the encoder-decoder structure involved in computer vision tasks. The initial image processing model includes at least an image coding network (such as an image encoder), and the network adjustment layer can be a fully connected layer.

[0120] Schematic illustration: The service platform adds at least one model network adjustment layer to the initial image processing model to obtain a reference image processing model. In specific implementation: at least one fully connected layer can be connected to the image coding network in the initial image processing model. For example, connecting several fully connected layers to the image coding network yields the reference image processing model. The structurally adjusted reference image processing model is then trained using the aforementioned sample pairs to adjust its structural parameters.

[0121] S208: The reference image processing model is fine-tuned using the normal image processing samples to generate an image processing model.

[0122] Understandably, in this specification, only the reference image processing model with the model structure adjusted is used to adapt to the new scene. In the subsequent training phase for the reference image processing model, the parameters of the aforementioned model network adjustment layer are usually updated, while the parameters of the rest remain unchanged.

[0123] To illustrate, taking a reference image processing model as an example with an encoder-decoder structure, typically only the parameters of the adjusted model network layers (such as fully connected layers) in the image coding network are updated, while the parameters of other model layers in the image coding network remain unchanged.

[0124] It is understood that the image processing normal sample pair is the image processing normal sample pair that meets the expectations after verifying and screening several image processing sample pairs. The image processing normal sample pair includes reference input image data and reference model output data.

[0125] In one feasible implementation, image processing anomalous sample pairs can also be included in the model fine-tuning process. By labeling the image processing anomalous sample pairs as negative examples, the model is trained based on the labeled negative example image processing anomalous sample pairs as a reference image processing model to obtain a trained image processing model.

[0126] 1. The reference image processing model is used to extract features from the output data of the reference model to obtain image reference features.

[0127] Schematic example, taking a reference image processing model as an encoder-decoder structure, the reference image processing model may include at least an image decoding network (such as a decoder) and an image encoding network (encoder);

[0128] The service platform extracts features from the output data of the reference image processing model to obtain image reference features, which can specifically be:

[0129] The service platform adds noise to the reference model output data to obtain the processed reference model output data.

[0130] The service platform uses an image decoding network and backpropagation to extract features from the output data of the reference model, thereby obtaining image reference features.

[0131] In normal image processing samples, for reference input image data and reference model output data 'a' that meet image processing specifications (such as privacy and security specifications), Gaussian noise is added to the reference model output data, and the corresponding image reference features 'f' are obtained by backpropagation through an image decoding network (such as a decoder). The above process can be expressed as:

[0132] f = decoder -1 (a)

[0133] Where f represents the image reference feature, a represents the reference model output data, and decoder -1 () indicates the decoding process of the image decoding network.

[0134] 2. Based on the reference input image data and the image reference features, training sample data is obtained.

[0135] Understandably, the reference input image data and the image reference features f are combined to form the training sample data for the image coding network.

[0136] 3. Based on the training sample data and the output data of the reference model, train the reference image processing model to obtain the trained image processing model.

[0137] Schematic, the reference image processing model includes at least an image encoding network and an image decoding network, and the training sample data includes reference input image data and the image reference features;

[0138] In one feasible implementation, the reference image processing model is trained based on training sample data to obtain a trained image processing model, specifically:

[0139] 3.1 During the training process of the image coding network, the service platform performs network coding training on the image coding network based on the reference input image data and image reference features, and determines the coding output features of the image coding network for the reference input image data; that is, the reference input image data is input into the reference image processing model, and the image coding network of the reference image processing model is trained by forward propagation, and then the coding output features for the reference input image data are output. Then, backpropagation training is used to calculate the coding loss based on the image reference features and the coding output features, and the model parameters are adjusted based on the coding loss until the training is completed.

[0140] Indicatively, the service platform uses an encoding loss function to update the network parameters of the model network adjustment layer in the image encoding network based on the reference input image data and the image reference features, thus obtaining the trained image encoding network. In specific implementation: the reference input image data is input into the model and processed by the image encoding network to obtain the encoded output features. Then, the encoding loss between the encoded output features and the image reference features is calculated according to the encoding loss function. Based on the encoding loss, the parameters of the model network adjustment layer (such as a fully connected layer) are adjusted. For example, the parameters include the weight values ​​and thresholds of the model network adjustment layer, until the entire reference image processing model's image encoding network reaches the end-of-training condition and the training is completed, and the model encoding network converges.

[0141] The encoding loss function satisfies the following formula:

[0142]

[0143] Wherein, Loss A is the encoding loss, f is the image reference feature, and f′ is the encoded output feature of the image encoding network for the reference input image data.

[0144] Optionally, the encoding loss function takes the image reference feature f and the encoded output feature f′ as inputs and calculates the distance (such as Euclidean distance) as the decoding loss.

[0145] Furthermore, the process of training the reference image processing model includes the training of the image encoding network and the training of the image decoding network. The training process of the image decoding network is explained below:

[0146] 3.2 During the training of each round of the image coding network, the encoded output features f′ of the image coding network for the reference input image data are used as the input of the image decoding network. The image coding network performs image decoding processing on the encoded output features f′ and outputs decoded output data I′. Then, the decoding loss is calculated based on the reference model output data I and the decoded output data I′. The image decoding network is trained based on the decoding loss until the entire reference image processing model's image decoding network meets the training termination condition of the coding network and the training is completed. At this point, the trained image processing model can be obtained.

[0147] As an illustration, the service platform uses a decoding loss function to train the image decoding network based on the encoded output features and image reference features, and obtains the trained image decoding network.

[0148] The decoding loss function satisfies the following formula:

[0149]

[0150] Wherein, Loss B is the decoding loss, I is the output data of the reference model, and I′ is the decoding output data of the image decoding network.

[0151] Optionally, the decoding loss function takes the reference model output data I and the decoded output data I′ as inputs and calculates the negative distance (such as Euclidean distance) as the decoding loss.

[0152] 3.3 Determine the training termination conditions that satisfy the image processing model to obtain the trained image processing model.

[0153] Yes, the process of training the reference image processing model includes the training of the image encoding network and the image decoding network. The trained image processing model is obtained when both the image encoding network and the image decoding network training processes meet the conditions for ending the training.

[0154] In one or more embodiments of this specification, the model termination condition may include, for example, the value of the loss function being less than or equal to a preset loss function threshold, or the number of iterations reaching a preset threshold. Specific model termination conditions can be determined based on actual circumstances and will not be elaborated here.

[0155] S210: Perform transaction processing on the target transaction scenario based on the image processing model.

[0156] For details, please refer to the method steps in other embodiments of this specification, which will not be repeated here.

[0157] In one or more embodiments of this specification, the service platform deploys an initial image processing model to at least one client in the target transaction scenario for model trial operation. The client processes the input image data in the target transaction scenario based on the initial image processing model to obtain model output data. Based on this, the service platform can collect at least one image processing sample pair from the client for the initial image processing model, and then use the image processing sample pair to perform model calibration on the initial image processing model to generate an image processing model. Then, the service platform can perform transaction processing on the target transaction scenario based on this image processing model. The calibrated image processing model can resist the distribution differences between training data and actual online data, has good model robustness, optimizes the model processing process, and effectively enhances the stability and scenario generalization ability of the model after being deployed to the target transaction scenario.

[0158] Please see Figure 4 , Figure 4 This is a flowchart illustrating another embodiment of a model processing method proposed in one or more embodiments of this specification.

[0159] To better understand the model processing methods designed in this specification, the following uses a privacy-de-identification scenario as the target transaction scenario and an image de-identification model as the image processing model as an example to explain the model processing methods involved in this specification. This embodiment and one or more model processing method embodiments belong to the same technical concept. For technical descriptions not covered in this embodiment, please refer to one or more model processing method embodiments. Specifically:

[0160] In recent years, biometric technology has been widely applied to people's production and daily life. For example, facial recognition payment, facial access control, facial attendance, and facial recognition for station entry all rely on biometrics. However, with the increasingly widespread application of biometric technology, its security issues have become increasingly prominent. To achieve biometric authentication, biometric systems often need to collect, process, transmit, and store users' biometric information. Each step in this process has the potential to leak users' biometric information (highly private information), potentially leading to further financial and informational losses. Therefore, protecting users' biometric privacy information is a common scenario involving privacy desensitization. Privacy desensitization involves using image processing models to desensitize biometric images, resulting in a desensitized image that visually completely lacks the user's private characteristics, thus achieving the purpose of privacy protection.

[0161] S302: Deploy the initial image processing model to at least one client in the target transaction scenario.

[0162] Understandably, the initial image processing model is the initial image desensitization model.

[0163] In a schematic way, the service platform pre-builds an initial machine learning model offline for privacy desensitization scenarios. The initial machine learning model is set based on the image privacy protection task to adapt to the privacy desensitization scenario. The initial machine learning model is trained offline using offline image sample data. When the offline model termination condition is met, the trained target image processing model, i.e. the target image desensitization model, is obtained. Then, the target image desensitization model is used as the initial image desensitization model and the initial image desensitization model is deployed online.

[0164] The privacy protection mentioned above can be understood as desensitizing biometric images at the visual / semantic level to filter out privacy information.

[0165] Indicatively, the service platform can deploy the initial image desensitization model to at least one client associated with the target transaction scenario, so that the client can process the input image data in the target transaction scenario based on the initial image desensitization model to obtain model output data;

[0166] S304: Obtain encrypted sample processing data for the initial image processing model from the client. The encrypted sample processing data is encrypted data of at least one image processing sample pair, which includes input image data and model output data.

[0167] Accumulation of transaction scenario data: For a new privacy desensitization transaction scenario, firstly, deploy several clients corresponding to the service platform, then launch the initial image desensitization model, and start a trial operation for a period of time based on the initial image desensitization model. During this stage, the initial image desensitization model usually does not have cold start characteristics due to objective factors and cannot be directly used in long-term privacy desensitization transaction scenarios.

[0168] Encrypted sample data upload: During this trial operation phase (e.g., 24-48 hours), the client will use the user's original biometric image as input image data and perform image desensitization using an initial image desensitization model to obtain a privacy-protected image, which is also the model output data. The client user's original biometric image and privacy-protected image are used as an image processing sample pair and encrypted simultaneously (to ensure basic privacy and security) to obtain encrypted sample processing data. The encrypted sample processing data consists of encrypted data of at least one image processing sample pair.

[0169] Encrypted data acquisition: Several clients can upload these encrypted sample processing encrypted data to the service platform, and the service platform can then obtain the sample processing encrypted data for the initial image processing model from the clients.

[0170] Understandably, this trial operation phase (e.g., 24-48 hours) is a trial operation of the initial image desensitization model. Due to objective factors, the data interaction traffic is small and the operation time is usually short. Therefore, conventional encryption privacy protection methods are used to ensure privacy and security. After the service platform generates the image processing model and completes the cold start, it can deploy the privacy protection method based on deep learning, which means that the image processing model will be officially launched in privacy desensitization scenarios.

[0171] The cold start problem, in one or more embodiments of this specification, refers to the problem of deploying a privacy-preserving model to a new scenario and adapting it to the new scenario;

[0172] In one or more embodiments of this specification, "small sample" refers to a small amount of original image data that meets privacy protection performance requirements in a new privacy desensitization scenario, i.e., sample processing encrypted data.

[0173] S306: Decrypt the encrypted data of the sample processing to obtain at least one image processing sample, and perform model calibration processing on the initial image processing model based on the at least one image processing sample to generate an image processing model;

[0174] 1. Perform verification and screening on at least one image processing sample pair to obtain at least one normal image processing sample pair and one abnormal image processing sample pair;

[0175] The verification and screening process can be understood as follows: the service platform verifies the input results of the image processing samples. By judging whether each set of input image data and model output data meets expectations, that is, whether the model output data output by the initial image processing model for the input image data meets expectations, the normal image processing sample pairs that meet expectations and those that do not meet expectations can be distinguished.

[0176] Verification and filtering can also be understood as data annotation: data decryption is performed on the server side. The image processing sample pairs contain the original image and the corresponding privacy-protected image. It is determined whether the privacy-protected image meets the privacy protection requirements. Data sets that meet the privacy protection requirements are distinguished as normal image processing sample pairs, and data sets that do not meet the privacy protection requirements are distinguished as abnormal image processing sample pairs.

[0177] 2. Based on the normal image processing sample pairs and the abnormal image processing sample pairs, the initial image processing model is structurally adjusted to obtain a reference image processing model.

[0178] The service platform can evaluate the performance of the initial privacy-de-identification model in real time based on the distribution of the number of normal and abnormal image processing samples. Based on the performance evaluation, the platform can then restructure the initial privacy-de-identification model and train it using samples. For example, this could involve calculating the cumulative rate.

[0179] Cumulative rate calculation: The number of abnormal image processing samples N1 that do not meet privacy protection requirements and the number of normal image processing sample pairs N2 that meet privacy protection requirements are counted, along with the trial operation time t corresponding to the collection of these samples. Then, the cumulative rate of the samples is calculated. The higher this rate, the worse the performance of the current privacy protection model in the new scenario.

[0180] The service platform determines the model network adjustment layers for the initial privacy-de-identifying model based on the sample accumulation rate;

[0181] Schematic, the evaluation reference threshold may include at least a first reference threshold and a second reference threshold, specifically determining at least one model network adjustment layer for the initial image processing model.

[0182] Optionally, the model network adjustment layer can be a model network layer in the initial privacy desensitization model, or a model network layer that needs to be added to the original network structure of the initial image processing model. The type of model network layer can be a fit of one or more of the network layer types corresponding to the machine learning models involved in related technologies, such as a fully connected layer (FC layer).

[0183] For example, suppose the first reference threshold is denoted as v1, and the second reference threshold is denoted as v2(v1). <v2)。

[0184] If the performance evaluation parameter - the cumulative sample rate v is less than v1, then the model network adjustment layer for the initial privacy desensitization model is determined to be the first number. For example, the first number can be one, or it can be two FC layers after the image coding network (such as the image encoder).

[0185] If the performance evaluation parameter – the sample accumulation rate v – is between v1 and v2, then the model network adjustment layers for the initial privacy-de-identifying model are determined to be the second number. For example, the second number could be two, such as two fully connected (FC) layers following an image coding network (e.g., an image encoder).

[0186] If the performance evaluation parameter - the cumulative sample rate v - is greater than v2, then the model network adjustment layers for the initial privacy-de-identification model are determined to be a third number. For example, the third number can be three, such as three fully connected (FC) layers following an image coding network (e.g., an image encoder).

[0187] Furthermore, the service platform then performs structural adjustment processing on the initial privacy-de-identification model based on the model network adjustment layer to obtain a reference privacy-de-identification model.

[0188] Indicatively, the initial privacy desensitization model can be a model constructed from the encoder-decoder structure involved in the computer vision task. The initial privacy desensitization model includes at least an image coding network (such as an image encoder), and the network adjustment layer of the above model can be a fully connected layer.

[0189] 3. The service platform then uses the image processing normal samples to fine-tune the reference privacy desensitization model to generate a privacy desensitization model.

[0190] The model fine-tuning process can be referred to in other method embodiments of this specification, which will not be repeated here.

[0191] S308: Encrypt the image processing model data to obtain encrypted model data;

[0192] Understandably, after the image processing model, i.e. the privacy-de-identifying model, is trained, it is then encrypted using an encryption algorithm to obtain encrypted model data. This is to complete the cold start deployment in the new scenario.

[0193] S310: Send the encrypted model data to at least one client so that the client can perform transaction processing based on the image processing model after the encrypted model data has been decrypted.

[0194] Understandably, for a large number of clients (such as all clients) deployed in a new scenario, the trained privacy-de-identified model is encrypted. After encryption, the service platform distributes the encrypted model to the corresponding client for formal online deployment. On the client, the encrypted data is decrypted to obtain the privacy-de-identified model. Subsequently, this privacy-de-identified model is used for privacy protection, which can achieve better privacy protection performance in the corresponding new scenario.

[0195] In a specific implementation scenario, the service platform can instruct several clients to input privacy images in the privacy-de-identification transaction scenario into the image de-identification model. The image de-identification model is used to perform visual / semantic de-identification on the privacy images to filter out privacy information, thereby outputting a transaction-de-identified image.

[0196] The privacy image can be user images related to privacy and security, biometric images (such as facial images, fingerprint images, limb images), photographs, communication record images, meeting images, biometric images, etc. The specific type of privacy image can be set based on the actual business scenario, and is not specifically limited here.

[0197] The transaction-desensitized image is the output transaction-desensitized image after the privacy image has been desensitized at the visual / semantic level through an image desensitization model to filter out privacy information.

[0198] In one or more embodiments of this specification, the service platform conducts model trial runs by deploying an initial image processing model to at least one client in a privacy-desensitized transaction scenario. The client processes the input image data in the target transaction scenario based on the initial image processing model to obtain model output data. Based on this, the service platform can collect at least one image processing sample pair from the client for the initial image processing model, and then use the image processing sample pair to perform model calibration on the initial image processing model to generate an image processing model. This image processing model can then be used to process transactions in the privacy-desensitized transaction scenario. The calibrated image processing model can resist the distribution differences between training data and actual online data, exhibiting good model robustness. It optimizes the model processing process, effectively enhancing the stability and scenario generalization ability of the model after deployment in the privacy-desensitized transaction scenario. Furthermore, it solves the cold start phenomenon of the offline-trained initial image processing model after deployment, realizing a new scenario cold start based on small-sample model recoding training originating from the client during the trial run phase. In the privacy-desensitized transaction scenario, it can avoid privacy information leakage and improve privacy protection performance.

[0199] Please see Figure 5 This document provides a flowchart illustrating a model processing method according to one or more embodiments of this specification. This method can be implemented using a computer program and can run on a background investigation device based on the von Neumann architecture. The computer program can be integrated into an application or run as a standalone utility application. The model processing device can be a client application.

[0200] S402: Obtain the initial image processing model from the service platform and deploy the initial image processing model to the target transaction scenario;

[0201] According to some embodiments, the service platform pre-builds an initial machine learning model offline for the target transaction scenario. The initial machine learning model is set based on image processing tasks such as image recognition, image classification, and image privacy protection to adapt to the target transaction scenario. The initial machine learning model is trained offline using offline image sample data. When the offline model termination condition is met, a trained target image processing model is obtained. Then, the target image processing model is used as the initial image processing model and deployed online.

[0202] As an illustration, the service platform can deploy the initial image processing model to at least one client associated with the target transaction scenario. The client can then obtain the initial image processing model from the service platform and process the input image data in the target transaction scenario based on the initial image processing model to obtain the model output data.

[0203] Optionally, by constructing an online transaction testing environment containing at least one client, with a target number (e.g., 100) of clients as model testing devices, the generated image processing model can be deployed online on a large scale after the initial image processing model has been calibrated by the online training model, such as distributing the generated image processing model to all clients associated with the target transaction scenario.

[0204] S404: Based on the initial image processing model, perform model processing on the input image data to obtain model output data;

[0205] Furthermore, the client can apply the initial image processing model to the target transaction scenario for image processing. In this scenario, input image data will be generated, specifically the input data for the initial image processing model. The client will then input this data into the initial image processing model for processing, and the model will output its output data. In other words, the initial image processing model is used to process the input image data from the target transaction scenario to obtain the model's output data.

[0206] S406: Generate image processing sample pairs based on the input image data and model output data, and send the image processing sample pairs to the service platform so that the service platform can perform model calibration processing on the initial image processing model based on at least one of the image processing sample pairs to generate an image processing model, and perform transaction processing on the target transaction scenario based on the image processing model.

[0207] Understandably, the client collects at least one set of input image data and model output data for the initial image processing model; this set of input image data and model output data constitutes an image processing sample pair. The number of image processing sample pairs can be multiple.

[0208] As an illustration, the client can also actively collect image processing sample pairs and send them to the service platform. In this case, the service platform can receive at least one image processing sample pair sent by the client. These image processing sample pairs include input image data and model output data.

[0209] Furthermore, the service platform will perform model calibration on the initial image processing model based on at least one image processing sample to generate an image processing model. After the image processing model is generated, it can be deployed online and adapted to the target transaction scenario.

[0210] Understandably, the service platform can deploy the image processing model to a large number of clients associated with the target transaction scenario. At this time, the clients can accurately process the input image data generated online in the target transaction scenario based on the image processing model to obtain model output data with good processing effect.

[0211] In one or more embodiments of this specification, the service platform deploys an initial image processing model to at least one client in the target transaction scenario for model trial operation. The client processes the input image data in the target transaction scenario based on the initial image processing model to obtain model output data. Based on this, the service platform can collect at least one image processing sample pair from the client for the initial image processing model, and then use the image processing sample pair to perform model calibration on the initial image processing model to generate an image processing model. Then, the service platform can perform transaction processing on the target transaction scenario based on this image processing model. The calibrated image processing model can resist the distribution differences between training data and actual online data, has good model robustness, optimizes the model processing process, and effectively enhances the stability and scenario generalization ability of the model after being deployed to the target transaction scenario.

[0212] The following will combine Figure 6 This manual provides a detailed description of the model processing device provided. It should be noted that... Figure 6 The model processing apparatus shown is used to execute the methods of one or more model processing methods shown in this specification. For ease of explanation, only the parts related to this specification are shown. For specific technical details not disclosed, please refer to the embodiments shown in one or more model processing methods shown in this specification.

[0213] Please see Figure 6 This diagram illustrates the structure of the model processing device described in this specification. The model processing device 1 can be implemented as all or part of a user terminal through software, hardware, or a combination of both. According to some embodiments, the model processing device 1 includes a model processing module 11, a model processing module 12, and a model processing module 13, specifically used for:

[0214] The sample acquisition module 11 is used to deploy the initial image processing model to at least one client in the target transaction scenario, and to acquire at least one image processing sample pair for the initial image processing model from the client. The image processing sample pair includes input image data and model output data.

[0215] Model generation module 12 is used to perform model calibration processing on the initial image processing model based on the at least one image processing sample pair to generate an image processing model;

[0216] The model update module 13 is used to perform transaction processing on the target transaction scenario based on the image processing model.

[0217] Optional, such as Figure 7 As shown, the sample acquisition module 11 includes:

[0218] Offline training unit 111 is used to train the model offline for the target transaction scenario to obtain a trained reference image processing model.

[0219] The model deployment unit 112 is used to deploy the reference image processing model as an initial image processing model to at least one client in the target transaction scenario, so that the client processes the input image data in the target transaction scenario based on the initial image processing model to obtain model output data;

[0220] The sample acquisition unit 113 is used to receive at least one image processing sample pair sent by the client, the image processing sample pair including the input image data and the model output data.

[0221] Optional, such as Figure 8 As shown, the model generation module 12 includes:

[0222] The verification and filtering unit 121 is used to perform verification and filtering on the at least one image processing sample pair to obtain at least one normal image processing sample pair and one abnormal image processing sample pair.

[0223] The structural adjustment unit 122 is used to perform structural adjustment processing on the initial image processing model based on the image processing normal sample pair and the image processing abnormal sample pair to obtain a reference image processing model.

[0224] The model fine-tuning unit 123 is used to perform model fine-tuning on the reference image processing model using the normal image processing samples to generate an image processing model.

[0225] Optionally, the verification and filtering unit 121 is specifically used for:

[0226] The at least one image processing sample pair is verified and screened based on the image processing specifications to determine at least one normal image processing sample pair that matches the image processing specifications, and at least one normal image processing sample pair that does not match the image processing specifications.

[0227] Optional, such as Figure 9As shown, the structural adjustment unit 122 is used for:

[0228] The parameter determination subunit 1221 is used to determine the performance evaluation parameters for the initial image processing model based on the normal image processing sample pairs and the abnormal image processing sample pairs.

[0229] The structural adjustment subunit 1222 is used to determine the model network adjustment layer for the initial image processing model based on the performance evaluation parameters;

[0230] The structure adjustment subunit 1222 is also used to perform structure adjustment processing on the initial image processing model based on the model network adjustment layer to obtain a reference image processing model.

[0231] Optionally, the parameter determining subunit 1221 is specifically used for

[0232] Obtain a first number of abnormal image processing sample pairs and a second number of normal image processing sample pairs;

[0233] Determine the model runtime of the initial image processing model;

[0234] Based on the first quantity, the second quantity, and the model running time, the sample accumulation rate is determined, and the sample accumulation rate is used as a performance evaluation parameter for the initial image processing model.

[0235] The cumulative rate of the samples conforms to the following formula:

[0236] v=(N1 / N2) / t

[0237] Where v is the cumulative rate of the samples, N1 is the first quantity, N2 is the second quantity, and t is the model running time.

[0238] Optional, structural adjustment subunit 1222, specifically used for:

[0239] Obtain at least one evaluation reference threshold for the performance evaluation parameter;

[0240] Based on the at least one evaluation reference threshold and the performance evaluation parameters, at least one model network adjustment layer is determined for the initial image processing model;

[0241] The structure adjustment subunit 1222 is further configured to: add the at least one model network adjustment layer to the initial image processing model to obtain a reference image processing model.

[0242] Optionally, the at least one evaluation reference threshold includes at least a first reference threshold and a second reference threshold, and the structural adjustment subunit 1222 is specifically used for:

[0243] If the performance evaluation parameter is less than the first reference threshold, then a first number of model network adjustment layers are determined;

[0244] If the performance evaluation parameter is greater than or equal to the first reference threshold and less than or equal to the second reference threshold, then a second number of model network adjustment layers are determined.

[0245] If the performance evaluation parameter is greater than the second reference threshold, then a third number of model network adjustment layers are determined.

[0246] Optionally, the initial image processing model includes at least an image coding network, the model network adjustment layer is a fully connected layer, and the structure adjustment subunit 1222 is specifically used to: connect the at least one fully connected layer to the image coding network to obtain a reference image processing model.

[0247] Optionally, the image processing normal sample pair includes reference input image data and reference model output data, and the model fine-tuning unit 123 is used for:

[0248] Image reference features are obtained by extracting features from the output data of the reference image processing model.

[0249] Training sample data is obtained based on the reference input image data and the image reference features;

[0250] The reference image processing model is trained based on the training sample data and the output data of the reference model to obtain the trained image processing model.

[0251] Optionally, the reference image processing model includes at least an image decoding network, and the model fine-tuning unit 123 is specifically used to: add noise to the output data of the reference model to obtain the processed output data of the reference model;

[0252] The image decoding network uses backpropagation to extract features from the output data of the reference model, thereby obtaining image reference features.

[0253] Optionally, the image processing model includes at least an image encoding network and an image decoding network, the training sample data includes reference input image data and the image reference features, and the model fine-tuning unit 123 is used for:

[0254] The image coding network is trained based on the reference input image data and the image reference features, and the coding output features of the image coding network for the reference input image data are determined.

[0255] The image decoding network is trained based on the encoded output features and the output data of the reference model.

[0256] Determine the model termination conditions that satisfy the image processing model to obtain the trained image processing model.

[0257] Optionally, the model fine-tuning unit 123 is used to: perform network parameter update training on the model network adjustment layer in the image coding network based on the reference input image data and the image reference features using an encoding loss function, so as to obtain the trained image coding network;

[0258] The encoding loss function satisfies the following formula:

[0259]

[0260] Wherein, Loss A is the encoding loss, f is the image reference feature, and f′ is the encoded output feature of the image encoding network for the reference input image data.

[0261] Optionally, the model fine-tuning unit 123 is used to: train the image decoding network using a decoding loss function based on the encoded output features and the reference model output data to obtain the trained image decoding network;

[0262] The decoding loss function satisfies the following formula:

[0263]

[0264] Wherein, Loss B is the decoding loss, I is the output data of the reference model, and I′ is the decoding output data of the image decoding network.

[0265] Optionally, the device 1 is specifically used for:

[0266] Obtain encrypted sample processing data for the initial image processing model from the client, wherein the encrypted sample processing data is encrypted data of at least one image processing sample pair;

[0267] The process of performing transaction processing on the target transaction scenario based on the image processing model includes:

[0268] The image processing model is encrypted to obtain encrypted model data;

[0269] The encrypted model data is sent to at least one client, so that the client can perform transaction processing based on the image processing model after the encrypted model data has been decrypted.

[0270] Optionally, the target transaction scenario is a privacy-de-identification transaction scenario, and the image processing model is an image de-identification model; the device 1 is specifically used for:

[0271] The privacy-de-identified image in the aforementioned privacy-de-identified transaction scenario is input into the image de-identification model, and the transaction-de-identified image is output.

[0272] It should be noted that the model processing apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when executing the model processing method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the model processing apparatus and the model processing method embodiments provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0273] The serial numbers in this specification are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0274] In this specification, the service platform conducts model trial runs by deploying an initial image processing model to at least one client in the target transaction scenario. The client processes the input image data in the target transaction scenario based on the initial image processing model to obtain model output data. Based on this, the service platform can collect at least one image processing sample pair from the client for the initial image processing model. Then, it uses the image processing sample pair to perform model calibration on the initial image processing model to generate a new image processing model. This new model can then be used to perform transaction processing on the target transaction scenario. The calibrated image processing model can withstand the distribution differences between training data and actual online data, exhibiting good model robustness. It optimizes the model processing process and effectively enhances the stability and scenario generalization ability of the model after deployment to the target transaction scenario.

[0275] Please see Figure 10 This diagram illustrates the structure of another model processing device described in this specification. This model processing device 2 can be implemented as all or part of a user terminal through software, hardware, or a combination of both. According to some embodiments, the model processing device 2 includes a sample acquisition module 21, a model generation module 22, and a model update module 23, specifically used for:

[0276] The sample acquisition module 21 is used to deploy the initial image processing model to at least one client in the target transaction scenario, and to acquire at least one image processing sample pair for the initial image processing model from the client. The image processing sample pair includes input image data and model output data. The initial image processing model is an image processing model trained offline for the online transaction scenario model.

[0277] Model generation module 22 is used to perform model calibration processing on the initial image processing model based on the at least one image processing sample pair to generate an image processing model;

[0278] The model update module 23 is used to perform transaction processing on the target transaction scenario based on the image processing model.

[0279] It should be noted that the model processing apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when executing the model processing method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the model processing apparatus and the model processing method embodiments provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0280] The serial numbers in this specification are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0281] In this specification, the service platform conducts model trial runs by deploying an initial image processing model to at least one client in the target transaction scenario. The client processes the input image data in the target transaction scenario based on the initial image processing model to obtain model output data. Based on this, the service platform can collect at least one image processing sample pair from the client for the initial image processing model. Then, it uses the image processing sample pair to perform model calibration on the initial image processing model to generate a new image processing model. This new model can then be used to perform transaction processing on the target transaction scenario. The calibrated image processing model can withstand the distribution differences between training data and actual online data, exhibiting good model robustness. It optimizes the model processing process and effectively enhances the stability and scenario generalization ability of the model after deployment to the target transaction scenario.

[0282] This specification also provides a computer storage medium capable of storing multiple instructions adapted to be loaded and executed by a processor as described above. Figures 1-5 The specific execution process of the model processing method described in the illustrated embodiment can be found in [reference needed]. Figures 1-5 The specific details of the illustrated embodiments will not be elaborated here.

[0283] This application also provides a computer program product storing at least one instruction, which is loaded and executed by the processor as described above. Figures 1-5 The specific execution process of the model processing method described in the illustrated embodiment can be found in [reference needed]. Figures 1-5The specific details of the illustrated embodiments will not be elaborated here.

[0284] Please see Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 11 As shown, the electronic device 1000 may include: at least one processor 1001, at least one network interface 1004, a user interface 1003, a memory 1005, and at least one communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or a camera; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The processor 1001 may include one or more processing cores. The processor 1001 connects to various parts within the server 1000 using various interfaces and lines, and performs various functions of the server 1000 and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1005, and by calling data stored in the memory 1005. Optionally, the processor 1001 can be implemented using at least one of the following hardware forms: Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 1001 can integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem can also be implemented as a separate chip without being integrated into the processor 1001.

[0285] The memory 1005 may include random access memory (RAM) or read-only memory. Optionally, the memory 1005 may include a non-transitory computer-readable storage medium. The memory 1005 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 1005 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 1005 may also be at least one storage device located remotely from the aforementioned processor 1001. Figure 11 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and application programs.

[0286] exist Figure 11 In the illustrated electronic device 1000, the user interface 1003 is mainly used to provide an input interface for the user and to obtain user input data; while the processor 1001 can be used to call the application program stored in the memory 1005 and specifically perform the following operations:

[0287] An initial image processing model is deployed to at least one client in the target transaction scenario, and at least one image processing sample pair for the initial image processing model is obtained from the client, the image processing sample pair including input image data and model output data;

[0288] Based on the at least one image processing sample pair, the initial image processing model is calibrated to generate an image processing model;

[0289] The target transaction scenario is processed based on the image processing model.

[0290] In one embodiment, when the processor 1001 executes the step of deploying the initial image processing model to at least one client in an online transaction scenario and obtaining at least one image processing sample pair for the initial image processing model from the client, it specifically performs the following steps:

[0291] The model is trained offline for the target transaction scenario to obtain a trained reference image processing model;

[0292] The reference image processing model is deployed as an initial image processing model to at least one client in the target transaction scenario, so that the client processes the input image data in the target transaction scenario based on the initial image processing model to obtain model output data;

[0293] Receive at least one image processing sample pair sent by the client, the image processing sample pair including the input image data and the model output data.

[0294] In one embodiment, when the processor 1001 performs model calibration processing on the initial image processing model based on the at least one image processing sample to generate an image processing model, it specifically performs the following steps:

[0295] The at least one image processing sample pair is subjected to verification and screening to obtain at least one normal image processing sample pair and one abnormal image processing sample pair.

[0296] Based on the normal image processing sample pairs and the abnormal image processing sample pairs, the initial image processing model is structurally adjusted to obtain a reference image processing model;

[0297] The reference image processing model is fine-tuned using the normal image processing samples to generate an image processing model.

[0298] In one embodiment, when the processor 1001 performs the verification and screening process on the at least one image processing sample pair to obtain at least one normal image processing sample pair and one abnormal image processing sample pair, it specifically performs the following steps:

[0299] The at least one image processing sample pair is verified and screened based on the image processing specifications to determine at least one normal image processing sample pair that matches the image processing specifications, and at least one normal image processing sample pair that does not match the image processing specifications.

[0300] In one embodiment, when the processor 1001 performs structural adjustment processing on the initial image processing model based on the image processing normal sample pairs and the image processing abnormal sample pairs to obtain a reference image processing model, it specifically performs the following steps:

[0301] Based on the normal image processing sample pairs and the abnormal image processing sample pairs, determine the performance evaluation parameters for the initial image processing model;

[0302] Based on the performance evaluation parameters, determine the model network adjustment layers for the initial image processing model;

[0303] The initial image processing model is structurally adjusted based on the model network adjustment layer to obtain a reference image processing model.

[0304] In one embodiment, when the processor 1001 determines the performance evaluation parameters for the initial image processing model based on the normal image processing sample pairs and the abnormal image processing sample pairs, it specifically performs the following steps:

[0305] Obtain a first number of abnormal image processing sample pairs and a second number of normal image processing sample pairs;

[0306] Determine the model runtime of the initial image processing model;

[0307] Based on the first quantity, the second quantity, and the model running time, the sample accumulation rate is determined, and the sample accumulation rate is used as a performance evaluation parameter for the initial image processing model.

[0308] The cumulative rate of the samples conforms to the following formula:

[0309] v=(N1 / N2) / t

[0310] Where v is the cumulative rate of the samples, N1 is the first quantity, N2 is the second quantity, and t is the model running time.

[0311] In one embodiment, when the processor 1001 performs the step of determining the model network adjustment layer for the initial image processing model based on the performance evaluation parameters, it specifically executes the following steps:

[0312] Obtain at least one evaluation reference threshold for the performance evaluation parameter;

[0313] Based on the at least one evaluation reference threshold and the performance evaluation parameters, at least one model network adjustment layer is determined for the initial image processing model;

[0314] The step of adjusting the structure of the initial image processing model based on the model network adjustment layer to obtain a reference image processing model includes:

[0315] The at least one model network adjustment layer is added to the initial image processing model to obtain a reference image processing model.

[0316] In one embodiment, the processor 1001, when performing the at least one evaluation reference threshold, includes at least a first reference threshold and a second reference threshold.

[0317] When determining at least one model network adjustment layer for the initial image processing model based on the at least one evaluation reference threshold and the performance evaluation parameters, the following steps are specifically performed:

[0318] If the performance evaluation parameter is less than the first reference threshold, then a first number of model network adjustment layers are determined;

[0319] If the performance evaluation parameter is greater than or equal to the first reference threshold and less than or equal to the second reference threshold, then a second number of model network adjustment layers are determined.

[0320] If the performance evaluation parameter is greater than the second reference threshold, then a third number of model network adjustment layers are determined.

[0321] In one embodiment, the initial image processing model includes at least an image coding network, and the model network adjustment layer is a fully connected layer. When the processor 1001 performs the step of adding the at least one model network adjustment layer to the initial image processing model to obtain a reference image processing model, it specifically performs the following steps:

[0322] The at least one fully connected layer is connected to the image coding network to obtain a reference image processing model.

[0323] In one embodiment, the normal image processing sample pair includes reference input image data and reference model output data. When the processor 1001 performs model fine-tuning processing on the reference image processing model using the normal image processing sample pair to generate an image processing model, it specifically performs the following steps:

[0324] Image reference features are obtained by extracting features from the output data of the reference image processing model.

[0325] Training sample data is obtained based on the reference input image data and the image reference features;

[0326] The reference image processing model is trained based on the training sample data and the output data of the reference model to obtain the trained image processing model.

[0327] In one embodiment, the reference image processing model includes at least an image decoding network. When the processor 1001 performs feature extraction on the output data of the reference model through the reference image processing model to obtain image reference features, it specifically performs the following steps:

[0328] The reference model output data is subjected to noise addition processing to obtain the processed reference model output data;

[0329] The image decoding network uses backpropagation to extract features from the output data of the reference model, thereby obtaining image reference features.

[0330] In one embodiment, the image processing model includes at least an image encoding network and an image decoding network, the training sample data includes reference input image data and the image reference features, and the processor 1001 performs the following steps when it executes the model training of the reference image processing model based on the training sample data and the reference model output data to obtain the trained image processing model:

[0331] The image coding network is trained based on the reference input image data and the image reference features, and the coding output features of the image coding network for the reference input image data are determined.

[0332] The image decoding network is trained based on the encoded output features and the output data of the reference model.

[0333] Determine the model termination conditions that satisfy the image processing model to obtain the trained image processing model.

[0334] In one embodiment, when the processor 1001 performs network coding training on the image coding network based on the reference input image data and the image reference features, it specifically executes the following steps:

[0335] Based on the reference input image data and the image reference features, the network parameters of the model network adjustment layer in the image coding network are updated and trained using the coding loss function to obtain the trained image coding network.

[0336] The encoding loss function satisfies the following formula:

[0337]

[0338] Wherein, Loss A is the encoding loss, f is the image reference feature, and f′ is the encoded output feature of the image encoding network for the reference input image data.

[0339] In one embodiment, when the processor 1001 performs network decoding training on the image decoding network based on the encoded output features and the reference model output data, it specifically executes the following steps:

[0340] Based on the encoded output features and the output data of the reference model, the image decoding network is trained using a decoding loss function to obtain the trained image decoding network.

[0341] The decoding loss function satisfies the following formula:

[0342]

[0343] Wherein, Loss B is the decoding loss, I is the output data of the reference model, and I′ is the decoding output data of the image decoding network.

[0344] In one embodiment, when the processor 1001 performs the step of obtaining at least one image processing sample pair for the initial image processing model from the client, it specifically executes the following steps:

[0345] Obtain encrypted sample processing data for the initial image processing model from the client, wherein the encrypted sample processing data is encrypted data of at least one image processing sample pair;

[0346] The process of performing transaction processing on the target transaction scenario based on the image processing model includes:

[0347] The image processing model is encrypted to obtain encrypted model data;

[0348] The encrypted model data is sent to at least one client, so that the client can perform transaction processing based on the image processing model after the encrypted model data has been decrypted.

[0349] In one embodiment, the target transaction scenario is a privacy-de-identified transaction scenario, and the image processing model is an image de-identification model; when the processor 1001 performs transaction processing on the target transaction scenario based on the image processing model, it specifically executes the following steps:

[0350] The privacy-de-identified image in the aforementioned privacy-de-identified transaction scenario is input into the image de-identification model, and the transaction-de-identified image is output.

[0351] Please refer to Figure 12 This diagram illustrates a structural block diagram of an electronic device provided in an exemplary embodiment of this application. The electronic device in this application may include one or more components such as a processor 110, a memory 120, an input device 130, an output device 140, and a bus 150. The processor 110, memory 120, input device 130, and output device 140 may be connected via the bus 150.

[0352] Processor 110 may include one or more processing cores. Processor 110 connects to various parts of the electronic device via various interfaces and lines, and performs various functions and processes data of electronic device 100 by running or executing instructions, programs, code sets, or instruction sets stored in memory 120, and by calling data stored in memory 120. Optionally, processor 110 may be implemented using at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). Processor 110 may integrate one or more of the following: central processing unit (CPU), graphics processing unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 110 and may be implemented separately through a communication chip.

[0353] The memory 120 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 120 may include a non-transitory computer-readable storage medium. The memory 120 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 120 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), instructions for implementing the various method embodiments described below, etc. The operating system may be the Android system, including systems deeply developed based on the Android system, the iOS system developed by Apple Inc., including systems deeply developed based on the iOS system, or other systems. The data storage area may also store data created by the electronic device during use, such as phonebook data, audio and video data, chat log data, etc.

[0354] See Figure 13As shown, the memory 120 can be divided into operating system space and user space. The operating system runs in the operating system space, while native and third-party applications run in the user space. To ensure that different third-party applications can achieve good running performance, the operating system allocates corresponding system resources for each application. However, different application scenarios within the same third-party application have different requirements for system resources. For example, in local resource loading scenarios, third-party applications have high requirements for disk read speed; in animation rendering scenarios, third-party applications have high requirements for GPU performance. Since the operating system and third-party applications are independent of each other, the operating system often cannot promptly perceive the current application scenario of a third-party application, resulting in the operating system's inability to adapt system resources accordingly to the specific application scenario of the third-party application.

[0355] In order for the operating system to distinguish the specific application scenarios of third-party applications, it is necessary to establish data communication between the third-party applications and the operating system. This would allow the operating system to obtain the current scenario information of the third-party applications at any time, and then perform targeted system resource adaptation based on the current scenario.

[0356] Taking the Android operating system as an example, the programs and data stored in memory 120 are as follows: Figure 14As shown, the memory 120 can store the Linux kernel layer 320, the system runtime library layer 340, the application framework layer 360, and the application layer 380. The Linux kernel layer 320, system runtime library layer 340, and application framework layer 360 belong to the operating system space, while the application layer 380 belongs to the user space. The Linux kernel layer 320 provides low-level drivers for various hardware components of the electronic device, such as display drivers, audio drivers, camera drivers, Bluetooth drivers, Wi-Fi drivers, and power management. The system runtime library layer 340 provides support for key features of the Android system through several C / C++ libraries. For example, the SQLite library provides database support, the OpenGL / ES library provides 3D graphics support, and the Webkit library provides browser kernel support. The system runtime library layer 340 also provides the Android runtime library, which mainly provides core libraries that allow developers to write Android applications using the Java language. The Application Framework Layer 360 provides various APIs that may be used when building applications. Developers can also use these APIs to build their own applications, such as activity management, window management, view management, notification management, content provider, package management, call management, resource management, and location management. At least one application runs in the Application Layer 380. These applications can be native applications that come with the operating system, such as contacts, SMS, clock, and camera apps; or third-party applications developed by third-party developers, such as games, instant messaging, and photo editing apps.

[0357] Taking the operating system as an example (iOS), the programs and data stored in memory 120 are as follows: Figure 15As shown, the iOS system includes: Core OS layer 420, Core Services layer 440, Media layer 460, and Cocoa Touch layer 480. Core OS layer 420 includes the operating system kernel, drivers, and low-level program frameworks. These low-level program frameworks provide hardware-level functionality for use by the program frameworks located in Core Services layer 440. Core Services layer 440 provides system services and / or program frameworks required by applications, such as Foundation framework, account framework, advertising framework, data storage framework, network connectivity framework, geolocation framework, motion framework, etc. Media layer 460 provides applications with audiovisual interfaces, such as interfaces related to graphics and images, audio technology, video technology, and AirPlay (wireless playback of audio and video transmission technologies). Cocoa Touch layer 480 provides various commonly used interface-related frameworks for application development and is responsible for user touch interaction on electronic devices. Examples include local notification services, remote push services, advertising frameworks, game tool frameworks, message user interface (UI) frameworks, UIKit frameworks, map frameworks, and so on.

[0358] exist Figure 15 The framework shown includes, but is not limited to, the base framework in the core service layer 440 and the UIKit framework in the touchable layer 480. The base framework provides many basic object classes and data types, offering the most basic system services to all applications, and is independent of the UI. The UIKit framework, on the other hand, provides a basic UI class library for creating touch-based user interfaces. iOS applications can use the UIKit framework to provide their UI, thus providing the application's infrastructure for building user interfaces, drawing, handling user interaction events, responding to gestures, and so on.

[0359] The methods and principles for implementing data communication between third-party applications and the operating system in the iOS system can be referenced from the Android system, and will not be elaborated here.

[0360] The input device 130 is used to receive input instructions or data, and includes, but is not limited to, a keyboard, mouse, camera, microphone, or touch device. The output device 140 is used to output instructions or data, and includes, but is not limited to, a display device and a speaker. In one example, the input device 130 and the output device 140 can be combined into a touch screen, which is used to receive touch operations from the user using a finger, stylus, or any suitable object on or near it, and to display the user interface of various applications. The touch screen is usually located on the front panel of the electronic device. The touch screen can be designed as a full-screen, curved screen, or irregularly shaped screen. The touch screen can also be designed as a combination of a full-screen and a curved screen, or a combination of an irregularly shaped screen and a curved screen; this specification does not limit this.

[0361] In addition, those skilled in the art will understand that the structure of the electronic device shown in the above figures does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, the electronic device may also include radio frequency circuits, input units, sensors, audio circuits, wireless fidelity (WiFi) modules, power supplies, Bluetooth modules, etc., which will not be described in detail here.

[0362] In this specification, the entity executing each step can be the electronic device described above. Optionally, the entity executing each step can be the operating system of the electronic device. The operating system can be Android, iOS, or other operating systems; this specification does not limit this.

[0363] The electronic device described in this manual may also be equipped with a display device. This display device can be any device capable of displaying information, such as a cathode ray tube display (CR), a light-emitting diode display (LED), an e-ink screen, a liquid crystal display (LCD), or a plasma display panel (PDP). Users can use the display device on electronic device 101 to view displayed text, images, videos, and other information. The electronic device may be a smartphone, tablet computer, gaming device, AR (Augmented Reality) device, automobile, data storage device, audio playback device, video playback device, laptop, desktop computing device, or wearable device such as an electronic watch, electronic glasses, electronic helmet, electronic bracelet, electronic necklace, or electronic clothing.

[0364] exist Figure 12 In the illustrated electronic device, which can be a terminal, the processor 110 can be used to call the application stored in the memory 120 and specifically perform the following operations:

[0365] Obtain an initial image processing model from the service platform and deploy the initial image processing model to the target transaction scenario;

[0366] Based on the initial image processing model, the input image data is processed to obtain the model output data;

[0367] Image processing sample pairs are generated based on the input image data and model output data. The image processing sample pairs are sent to the service platform so that the service platform can perform model calibration processing on the initial image processing model based on at least one of the image processing sample pairs to generate an image processing model, and perform transaction processing on the target transaction scenario based on the image processing model.

[0368] In one or more embodiments of this specification, the service platform deploys an initial image processing model to at least one client in the target transaction scenario for model trial operation. The client processes the input image data in the target transaction scenario based on the initial image processing model to obtain model output data. Based on this, the service platform can collect at least one image processing sample pair from the client for the initial image processing model, and then use the image processing sample pair to perform model calibration on the initial image processing model to generate an image processing model. Then, the service platform can perform transaction processing on the target transaction scenario based on this image processing model. The calibrated image processing model can resist the distribution differences between training data and actual online data, has good model robustness, optimizes the model processing process, and effectively enhances the stability and scenario generalization ability of the model after being deployed to the target transaction scenario.

[0369] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory, or random access memory, etc.

[0370] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A model processing method applied to a service platform, the method comprising: deploying an initial image processing model to at least one client in a target transaction scenario, obtaining at least one image processing sample pair for the initial image processing model from the client, the image processing sample pair comprising input image data and model output data; performing verification screening processing on the at least one image processing sample pair to obtain at least one image processing normal sample pair and an image processing abnormal sample pair, determining a performance evaluation parameter for the initial image processing model based on the image processing normal sample pair and the image processing abnormal sample pair, determining a model network adjustment layer for the initial image processing model based on the performance evaluation parameter, the performance evaluation parameter being used to quantify the degree of fit of the current initial image processing model and a new target transaction scenario; performing structural adjustment processing on the initial image processing model based on the model network adjustment layer to obtain a reference image processing model, performing model fine-tuning processing on the reference image processing model using the image processing normal sample pair to generate an image processing model; performing transaction processing on the target transaction scenario based on the image processing model.

2. The method of claim 1, wherein the deploying an initial image processing model to at least one client in an online transaction scenario, and obtaining at least one image processing sample pair for the initial image processing model from the client comprises: performing model offline training on a target transaction scenario to obtain a trained reference image processing model; deploying the reference image processing model as an initial image processing model to at least one client in a target transaction scenario, so that the client processes input image data in the target transaction scenario based on the initial image processing model to obtain model output data; receiving an image processing sample pair sent by at least one of the clients, the image processing sample pair comprising the input image data and the model output data.

3. The method of claim 1, wherein the performing verification screening processing on the at least one image processing sample pair to obtain at least one image processing normal sample pair and an image processing abnormal sample pair comprises: performing verification screening processing on the at least one image processing sample pair based on an image processing specification to determine at least one image processing normal sample pair that matches the image processing specification, and at least one image processing normal sample pair that does not match the image processing specification.

4. The method of claim 1, wherein the determining a performance evaluation parameter for the initial image processing model based on the image processing normal sample pair and the image processing abnormal sample pair comprises: obtaining a first quantity of the image processing abnormal sample pairs, and a second quantity of the image processing normal sample pairs; determining a model running time of the initial image processing model; determining a sample accumulation rate based on the first quantity, the second quantity, and the model running time, taking the sample accumulation rate as the performance evaluation parameter for the initial image processing model. The sample accumulation rate conforms to the following formula: v=(N1 / N2) / t Wherein, v is the sample accumulation rate, N1 is the first quantity, N2 is the second quantity, and t is the model running time.

5. The method of claim 1, wherein determining the model network adjustment layer for the initial image processing model based on the performance evaluation parameter comprises: obtaining at least one evaluation reference threshold for the performance evaluation parameter; determining at least one model network adjustment layer for the initial image processing model based on the at least one evaluation reference threshold and the performance evaluation parameter; wherein adjusting the structure of the initial image processing model based on the model network adjustment layer to obtain a reference image processing model comprises: adding the at least one model network adjustment layer to the initial image processing model to obtain a reference image processing model.

6. The method of claim 5, wherein the at least one evaluation reference threshold comprises at least a first reference threshold and a second reference threshold, wherein determining the model network adjustment layer for the initial image processing model based on the at least one evaluation reference threshold and the performance evaluation parameter comprises: if the performance evaluation parameter is less than the first reference threshold, determining a first number of model network adjustment layers; if the performance evaluation parameter is greater than or equal to the first reference threshold and less than or equal to the second reference threshold, determining a second number of model network adjustment layers; if the performance evaluation parameter is greater than the second reference threshold, determining a third number of model network adjustment layers.

7. The method of claim 5, wherein the initial image processing model comprises at least an image encoding network, and the model network adjustment layer is a fully connected layer, wherein adding the at least one model network adjustment layer to the initial image processing model to obtain a reference image processing model comprises: connecting the at least one fully connected layer to the image encoding network to obtain a reference image processing model.

8. The method of claim 1, wherein the image processing normal sample pair comprises reference input image data and reference model output data, wherein fine-tuning the reference image processing model to generate an image processing model comprises: extracting features from the reference model output data by the reference image processing model to obtain image reference features; obtaining training sample data based on the reference input image data and the image reference features; training the reference image processing model based on the training sample data and the reference model output data to obtain a trained image processing model.

9. The method of claim 8, wherein the reference image processing model comprises at least an image decoding network, wherein extracting features from the reference model output data by the reference image processing model to obtain image reference features comprises: adding noise to the reference model output data to obtain processed reference model output data; The image reference feature is obtained by performing feature extraction on the reference model output data by using a back propagation manner of the image decoding network.

10. The method of claim 9, wherein the image processing model comprises at least an image encoding network and an image decoding network, and the training sample data comprises reference input image data and the image reference feature. The reference image processing model is trained based on the training sample data and the reference model output data to obtain a trained image processing model, comprising: The image encoding network is trained based on the reference input image data and the image reference feature, and an encoding output feature of the image encoding network for the reference input image data is determined; The image decoding network is trained based on the encoding output feature and the reference model output data; A model end training condition of the image processing model is determined to obtain the trained image processing model.

11. The method of claim 10, wherein the image encoding network is trained based on the reference input image data and the image reference feature, comprising: The model network adjustment layer of the image encoding network is trained based on the reference input image data and the image reference feature by using an encoding loss function to update network parameters, and the trained image encoding network is obtained; The encoding loss function satisfies the following formula: wherein the Loss A is an encoding loss, f is an image reference feature, and f' is an encoding output feature of the image encoding network for the reference input image data.

12. The method of claim 10, wherein the image decoding network is trained based on the encoding output feature and the reference model output data, comprising: The image decoding network is trained based on the encoding output feature and the reference model output data by using a decoding loss function to obtain the trained image decoding network; The decoding loss function satisfies the following formula: wherein the Loss B is a decoding loss, I is the reference model output data, and I' is decoding output data of the image decoding network.

13. The method of claim 1, wherein the at least one image processing sample pair for the initial image processing model is obtained from the client, comprising: Sample processing encrypted data for the initial image processing model is obtained from the client, the sample processing encrypted data being encrypted data of the at least one image processing sample pair; The target transaction scenario is processed based on the image processing model, comprising: Model data of the image processing model is encrypted to obtain model encrypted data; The model encrypted data is sent to at least one client, so that the image processing model after decryption based on the model encrypted data is used for transaction processing by the client.

14. The method of any one of claims 1-13, wherein the target transaction scenario is a privacy desensitization transaction scenario, and the image processing model is an image desensitization model. The transaction processing of the target transaction scene based on the image processing model comprises: inputting the privacy image in the privacy desensitization transaction scene into the image desensitization model to output a transaction desensitization image.

15. A model processing method applied to a client, the method comprising: obtaining an initial image processing model from a service platform and deploying the initial image processing model to a target transaction scene; performing model processing on input image data based on the initial image processing model to obtain model output data; generating an image processing sample pair based on the input image data and the model output data, and sending the image processing sample pair to the service platform, so that the service platform performs model calibration processing on the initial image processing model based on at least one image processing sample pair to generate an image processing model, and performs transaction processing on the target transaction scene based on the image processing model; wherein the service platform performs model calibration processing on the initial image processing model based on at least one image processing sample pair to generate an image processing model, comprising: the service platform performing verification and screening processing on the at least one image processing sample pair to obtain at least one image processing normal sample pair and an image processing abnormal sample pair, determining a performance evaluation parameter for the initial image processing model based on the image processing normal sample pair and the image processing abnormal sample pair, the performance evaluation parameter being used to quantify the degree of fit between the current initial image processing model and the new target transaction scene, determining a model network adjustment layer for the initial image processing model based on the performance evaluation parameter, performing structural adjustment processing on the initial image processing model based on the model network adjustment layer to obtain a reference image processing model, and performing model fine-tuning processing on the reference image processing model using the image processing normal sample pair to generate an image processing model.

16. A model processing device, comprising: a sample obtaining module configured to deploy an initial image processing model to at least one client in a target transaction scene, and obtain at least one image processing sample pair for the initial image processing model from the client, the image processing sample pair comprising input image data and model output data; a model generating module configured to perform verification and screening processing on the at least one image processing sample pair to obtain at least one image processing normal sample pair and an image processing abnormal sample pair, determine a performance evaluation parameter for the initial image processing model based on the image processing normal sample pair and the image processing abnormal sample pair, the performance evaluation parameter being used to quantify the degree of fit between the current initial image processing model and the new target transaction scene, determine a model network adjustment layer for the initial image processing model based on the performance evaluation parameter, perform structural adjustment processing on the initial image processing model based on the model network adjustment layer to obtain a reference image processing model, and perform model fine-tuning processing on the reference image processing model using the image processing normal sample pair to generate an image processing model. The model updating module is configured to perform transaction processing on the target transaction scenario based on the image processing model.

17. A model processing apparatus, the apparatus comprising: a model deployment module configured to obtain an initial image processing model from a service platform, and deploy the initial image processing model to a target transaction scenario; a model processing module configured to perform model processing on input image data based on the initial image processing model to obtain model output data; a sample sending module configured to generate an image processing sample pair based on the input image data and the model output data, and send the image processing sample pair to the service platform, so that the service platform performs model calibration processing on the initial image processing model based on at least one image processing sample pair to generate an image processing model, and performs transaction processing on the target transaction scenario based on the image processing model; wherein the service platform performs model calibration processing on the initial image processing model based on at least one image processing sample pair to generate an image processing model, including: the service platform performs verification and screening processing on the at least one image processing sample pair to obtain at least one image processing normal sample pair and an image processing abnormal sample pair, determines a performance evaluation parameter for the initial image processing model based on the image processing normal sample pair and the image processing abnormal sample pair, the performance evaluation parameter being used to quantify the degree of fit between the current initial image processing model and a new target transaction scenario, determines a model network adjustment layer for the initial image processing model based on the performance evaluation parameter, performs structural adjustment processing on the initial image processing model based on the model network adjustment layer to obtain a reference image processing model, and performs model fine-tuning processing on the reference image processing model using the image processing normal sample pair to generate an image processing model.

18. A computer storage medium, the computer storage medium storing a plurality of instructions, the instructions being adapted to be loaded and executed by a processor to perform the method steps of any one of claims 1-14 or 15.

19. A computer program product, the computer program product storing at least one instruction, the at least one instruction being loaded and executed by a processor to perform the method steps of any one of claims 1-14 or 15.

20. An electronic device, comprising: a processor and a memory; wherein the memory stores a computer program, the computer program being adapted to be loaded and executed by the processor to perform the method steps of any one of claims 1-14 or 15.

Images