A dynamic model publishing method based on TensorFlow

Abstract

The present invention relates to the technical field of model publishing, and in particular to a TensorFlow-based model dynamic publishing method, comprising: deploying TensorFlow Serving; deploying a model; generating a model configuration, and publishing the model; determining whether the model publishing is qualified based on the time taken to generate the model; when it is determined that the model deployment is abnormal, adjusting the model deployment parameters based on a time deviation, wherein the deployment parameters include adjusting the allocated memory of a Docker container to a corresponding value, adjusting the TCP window to a corresponding value, or adjusting the compatibility detection period used for compatibility checking to a corresponding value; or determining that the model deployment is qualified, and generating an API address and a parameter template for the deployed model; monitoring the model deployment status, and when a model deployment abnormality is detected, adjusting the model generation parameters, thereby improving the model generation efficiency.

Description

Technical Field

[0001] The present invention relates to the field of model publishing technology, and in particular to a model dynamic publishing method based on TensorFlow. Background Art

[0002] Machine learning technology is becoming more and more mature. As an open source machine learning platform, TensorFlow provides functions such as data preprocessing and model training.

[0003] In actual machine learning and deep learning applications, it is often necessary to dynamically publish trained TensorFlow models so that they can be called and used in different environments.

[0004] How to uniformly deploy the trained models becomes an indispensable part.

[0005] Chinese patent application publication number: CN112799710A, discloses a model publishing system and model publishing method, including a model generation device, a model service forwarding device and a model publishing display device. The model generation device generates a to-be-published model according to a design strategy corresponding to a preset operating environment selected by an engineer, and sends the generated to-be-published model to the model service forwarding device; the model service forwarding device determines the publishing status corresponding to the received to-be-published model, and sends the determined publishing status corresponding to the to-be-published model to the model publishing display device. When the model publishing display device receives the corresponding publishing status, it displays the to-be-published model according to the corresponding publishing status; it can be seen that the above technical solution has the following problems: it does not take into account the monitoring of the model deployment status, and does not take into account the adjustment of the model generation parameters when an abnormality in model deployment is detected, which affects the model generation efficiency. Summary of the Invention

[0006] To this end, the present invention provides a model dynamic publishing method based on TensorFlow to overcome the problem that the existing technology does not consider monitoring the deployment of the model and does not consider adjusting the generation parameters of the model when an abnormality in the model deployment is detected, thereby affecting the generation efficiency of the model.

[0007] To achieve the above objectives, the present invention provides a model dynamic publishing method based on TensorFlow, comprising:

[0008] Deploy TensorFlow Serving;

[0009] Deploy the model;

[0010] Generate model configuration and publish the model;

[0011] The eligibility of a model release is determined based on the time it takes to generate the model, including:

[0012] Determine whether the model deployment is abnormal and adjust the model deployment parameters based on the duration deviation, where adjusting the model deployment parameters includes adjusting the allocated memory of the Docker container to a corresponding value, adjusting the TCP window to a corresponding value, or adjusting the compatibility detection period used for compatibility checking to a corresponding value;

[0013] Alternatively, determine whether the model is deployed properly and generate an API address and parameter template for the deployed model.

[0014] Furthermore, the process of determining whether the deployment of the model is qualified based on the monitored model release duration includes determining that the deployment of the model is abnormal when the release duration is greater than a second preset release duration, and adjusting the deployment parameters of the model based on the duration deviation.

[0015] Furthermore, the process of determining whether the deployment of the model is qualified based on the monitored model release time includes determining whether the deployment of the model is qualified based on the data volume of the model file when the release time is less than or equal to the second preset release time and greater than the first preset release time, and adjusting the first preset release time to a corresponding value based on the data volume when the data volume is greater than the preset data volume, wherein the increase in the first preset release time is proportional to the data volume.

[0016] Furthermore, the process of determining whether the deployment of the model is qualified based on the data volume of the model file also includes determining that the deployment of the model is abnormal when the data volume is less than or equal to the preset data volume, and adjusting the deployment parameters of the model based on the duration deviation.

[0017] Furthermore, the process of adjusting the deployment parameters of the model based on the duration deviation includes:

[0018] Determine the difference between the calculated release duration and the second preset release duration as a duration deviation;

[0019] When the duration deviation is less than or equal to a first preset duration deviation, adjusting the allocated memory of the Docker container to a corresponding value based on the model generation frequency of the server;

[0020] When the duration deviation is less than or equal to the second preset duration deviation and greater than the first preset duration deviation, adjusting the TCP window to a corresponding value based on the current data transmission rate;

[0021] When the duration deviation is greater than a second preset duration deviation, the compatibility detection period used for compatibility checking is adjusted to a corresponding value based on the abnormal frequency.

[0022] Furthermore, the allocated memory of the Docker container is adjusted to the corresponding value based on the model generation frequency of the server, where

[0023] The amount of allocated memory increases proportionally to the frequency of model generation.

[0024] Furthermore, the TCP window is adjusted to a corresponding value based on the current data transmission rate, wherein,

[0025] The increase in the TCP window is inversely proportional to the data transmission rate.

[0026] Furthermore, after the TCP window is adjusted, the compression ratio of the model file is adjusted to a corresponding value based on the difference between the adjusted TCP window and the initial TCP window, wherein,

[0027] The difference between the adjusted TCP window and the initial TCP window is recorded as the window difference;

[0028] The compression ratio increases in direct proportion to the window difference.

[0029] Furthermore, the compatibility detection period used for compatibility checking is adjusted to a corresponding value based on the abnormal frequency, wherein,

[0030] The reduction in the compatibility detection period is proportional to the anomaly frequency.

[0031] Furthermore, when the adjustment for the compatibility detection period is completed, the release duration is re-acquired, and when the re-acquired release duration is greater than the first preset release duration, the configuration file of TensorFlow Serving is updated.

[0032] Compared with the prior art, the beneficial effects of the present invention are: deploying TensorFlow Serving; deploying the model; generating the model configuration, and publishing the model; determining whether the model release is qualified based on the time taken to generate the model; when determining that the model deployment is abnormal, adjusting the model deployment parameters based on the time deviation, the deployment parameters include adjusting the allocated memory of the Docker container to a corresponding value, adjusting the TCP window to a corresponding value, or adjusting the compatibility detection period used for compatibility checking to a corresponding value; or determining that the model deployment is qualified, and generating an API address and parameter template for the deployed model; monitoring the model deployment status, and when detecting that the model deployment is abnormal, adjusting the model generation parameters, thereby improving the model generation efficiency.

[0033] Furthermore, deploying the TensorFlow Serving model provides a stable operating environment. The trained model is placed in the environment managed by TensorFlow Serving. A model configuration file is generated, informing TensorFlow Serving of the model's location, name, and platform information so that the model can be properly loaded and managed. The model's release qualification is determined based on the time it takes to generate the model. If the release time is less than or equal to the first preset release time, the model's loading and initialization operations can be completed quickly within the current timeframe. This indicates sufficient resources and good compatibility between the model and the environment, and the model deployment is considered qualified. If the release time is less than or equal to the second preset release time but greater than the first preset release time, the deployment qualification is further determined based on the data volume of the model file. Data volume affects model loading and initialization time, so a more detailed assessment based on data volume is required. If the release time is still long even when the data volume is less than or equal to the preset data volume, other abnormal factors are present, and the deployment is considered abnormal. If the data volume exceeds the preset data volume, the large data volume will extend the deployment time, and the first preset release time will be adjusted based on the data volume. Considering the impact of data volume on deployment time, we rationally evaluate model deployment. This improves the accuracy of model evaluation and further enhances model generation efficiency.

[0034] Furthermore, model deployment parameters are adjusted based on the duration deviation. The duration deviation reflects the extent to which model deployment exceeds normal time. The severity of the problem is determined based on the size of the duration deviation, and different adjustment strategies are implemented. If the duration deviation is less than or equal to a first preset duration deviation, the model deployment time is prolonged due to insufficient server resources. Therefore, the allocated memory for the Docker container is adjusted based on the server's model generation frequency. The server's model generation frequency indicates how busy the server is. When the model generation frequency is high, the server requires more resources to handle the generation and deployment of new models. By increasing the allocated memory for the Docker container, the server's processing capacity is improved to cope with the high frequency of model generation tasks. If the duration deviation is less than or equal to a second preset duration deviation and greater than the first preset duration deviation, the model deployment time is prolonged due to network transmission issues, or the data transmission rate is low due to network congestion or an improper TCP window setting. In this case, the TCP window size is adjusted based on the current data transmission rate, which indicates the network's transmission capacity. By adjusting the TCP window size, data transmission efficiency is optimized and the speed of model file transmission is increased. When the duration deviation is greater than the second preset duration deviation, in this case, the model deployment anomaly caused by compatibility issues causes the model release time to be too long. In this case, by reducing the compatibility detection cycle and performing compatibility checks more frequently, compatibility issues can be discovered and resolved in a timely manner, thereby further improving the model generation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a flowchart of the steps of the TensorFlow-based model dynamic publishing method according to an embodiment of the present invention;

[0036] Figure 2 A logic decision diagram for determining whether a model deployment is qualified based on the monitored model release duration according to an embodiment of the present invention;

[0037] Figure 3 A logical decision diagram for determining whether the deployment of a model is qualified based on the data volume of the model file according to an embodiment of the present invention;

[0038] Figure 4 This is a logic decision diagram for adjusting deployment parameters of a model based on duration deviation according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are merely used to explain the present invention and are not intended to limit the present invention.

[0040] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0041] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside", and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.

[0042] Furthermore, it should be noted that, in the description of the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0043] See also Figure 1 、 Figure 2 、 Figure 3 as well as Figure 4 As shown in the figure, they are respectively a flowchart of a method for dynamically publishing a model based on TensorFlow according to an embodiment of the present invention, a logical decision diagram for determining whether the model deployment is qualified based on the monitored model release duration, a logical decision diagram for determining whether the model deployment is qualified based on the data volume of the model file, and a logical decision diagram for adjusting the deployment parameters of the model based on the duration deviation; a method for dynamically publishing a model based on TensorFlow according to an embodiment of the present invention includes:

[0044] S1, deploy TensorFlow Serving;

[0045] S2, deploy the model;

[0046] S3, generates model configuration and publishes the model;

[0047] S4, based on the time taken to generate the model, determines whether the model is eligible for release, including:

[0048] Determine if the model deployment is abnormal and adjust the model deployment parameters based on the duration deviation. The deployment parameters include adjusting the allocated memory of the Docker container to a corresponding value, adjusting the TCP window to a corresponding value, or adjusting the compatibility detection period used for compatibility checking to a corresponding value.

[0049] Alternatively, determine whether the model is deployed properly and generate an API address and parameter template for the deployed model.

[0050] Specifically, in the process of publishing a model, you can select the model you want to publish on the page and dynamically generate its deployment configuration file based on Java.

[0051] Specifically, the code is as follows:

[0052] model_config_list {

[0053] config {

[0054] name: "mymodels" / / Specify the model name. This name is used to uniquely identify the model in TensorFlow Serving. When the client sends an inference request to TensorFlow Serving, it needs to use this name to specify the model to call.

[0055] base_path: " / root / python_soft / models / mymodel" / / Specify the base storage path of the model file. TensorFlow Serving will search for the model file in this path. The model files will be organized according to the version number. There can be multiple subdirectories under the base path, each subdirectory corresponding to a model version.

[0056] model_platform: "tensorflow" / / Specify the platform type used by the model. Setting this to "tensorflow" indicates that the model was trained using the TensorFlow framework. TensorFlow Serving will use this information to correctly load and process the model.

[0057] model_version_policy{

[0058] specific {

[0059] versions: 1

[0060] versions: 2} / / Specify the loading strategy of the model version, determine the model version that TensorFlowServing should load, use the specific strategy, and specify the specific version number.

[0061] / / Under the specific version strategy, explicitly specify the model version numbers to be loaded as 1 and 2. The client can choose to use either of these two versions when requesting inference.

[0062] }

[0063] }

[0064] }

[0065] Specifically, in S1, the deployment of TensorFlow Serving can be done locally in a Docker-based manner, including installing the Docker engine, pulling the official image of TensorFlow Serving from the Docker Hub, creating a Docker container, mounting the local model file into the container, and specifying the port mapping and environment variables when the container is started; optionally, in this embodiment, the Docker container allocates 3GB of memory.

[0066] Specifically, TensorFlow Serving can encapsulate the model trained using the TensorFlow framework into a form that can be provided to the outside world and deploy the model into a production environment.

[0067] Specifically, Docker containers achieve isolation, and applications running in Docker containers are isolated from the host machine and do not interfere with each other.

[0068] Specifically, in S1, the compatibility between TensorFlow Serving and the operating environment is periodically checked to ensure stable model generation and release.

[0069] Specifically, deploying the model in S2 can include saving the trained TensorFlow model in SavedModel format and placing it in a directory accessible to TensorFlow Serving. Ensure that the model's input and output formats meet TensorFlow Serving requirements.

[0070] Specifically, generating the model configuration in S3 and publishing the model can include creating a model configuration file, specifying the model name, version, and path, passing the model configuration file to the TensorFlow Serving service, starting the service, and loading the model.

[0071] Specifically, deploy TensorFlow Serving; deploy the model; generate the model configuration, and publish the model; determine whether the model release is qualified based on the time it takes to generate the model; when the model deployment is determined to be abnormal, adjust the model deployment parameters based on the time deviation, and the deployment parameters include adjusting the allocated memory of the Docker container to the corresponding value, adjusting the TCP window to the corresponding value, or adjusting the compatibility detection period used for compatibility checking to the corresponding value; or, determine whether the model deployment is qualified, and generate the API address and parameter template for the deployed model; monitor the model deployment status, and when a model deployment abnormality is detected, adjust the model generation parameters to improve the model generation efficiency.

[0072] Specifically, in S4, the process of determining whether the deployment of the model is qualified based on the monitored release duration of the model includes:

[0073] If the release duration is less than or equal to the first preset release duration, the model deployment is determined to be qualified, and an API address and parameter template are generated for the deployed model;

[0074] If the release duration is less than or equal to the second preset release duration and greater than the first preset release duration, determining whether the model deployment is qualified based on the data volume of the model file;

[0075] If the release duration is greater than the second preset release duration, the model deployment is determined to be abnormal, and the model deployment parameters are adjusted based on the duration deviation.

[0076] Specifically, the first preset release duration T1 is selected within the interval [55s, 65s], and the second preset release duration T2 is selected within the interval [80s, 86s].

[0077] It should be pointed out that the data in this embodiment are obtained by comprehensive analysis and evaluation of the historical detection data and the corresponding historical detection results in the four months before this detection. Before this detection, the present invention determines the values ​​of the various preset parameter standards for this detection based on the 2,857 actual models deployed in the first four months and the deployment time corresponding to each actual model, the actual abnormal situation of each actual model, the data transmission rate and the file size of the model. It can be understood by those skilled in the art that the present invention can determine the parameters of each single item by selecting the value with the highest proportion as the preset standard parameter based on the data distribution, using weighted summation to use the obtained value as the preset standard parameter or other selection methods, as long as the present solution can clearly define the different specific situations in the single determination process through the obtained values.

[0078] Specifically, through statistics and analysis, we determined that the deployment time for most simple to medium-complexity models under ideal conditions is concentrated around 55-65 seconds. Based on various abnormal situations encountered in actual deployments, multiple tests, and empirical analysis, we determined that the range of 80-86 seconds best defines these situations. It is understood that those skilled in the art may set these preset values ​​based on their actual implementation.

[0079] Specifically, the duration from when the TensorFlow Serving service is started to when the model is released is recorded as the model release duration.

[0080] Specifically, the process of determining whether a model is eligible for deployment based on the data volume of the model file includes:

[0081] If the data volume is less than or equal to the preset data volume, the model deployment is determined to be abnormal, and the model deployment parameters are adjusted based on the duration deviation;

[0082] If the data volume is greater than the preset data volume, the first preset publishing duration is adjusted to a corresponding value based on the data volume.

[0083] Specifically, the preset data volume S0 is selected within the range [100MB, 140MB].

[0084] Specifically, the data volume of the model file can be checked through file management tools or commands, and the folder where the model file is located can be found in the resource manager to determine the size of the model file. This is existing technology and will not be described in detail.

[0085] Specifically, based on the scale of statistical machine learning models, the file size of small to medium-sized models is between 100-140MB. Therefore, in this embodiment, S0 selects this interval as the threshold for judging the impact of data volume.

[0086] Specifically, deploying a TensorFlow Serving model provides a stable operating environment. The trained model is placed in the environment managed by TensorFlow Serving. A model configuration file is generated, informing TensorFlow Serving of the model's location, name, and platform information so that the model can be properly loaded and managed. The model's release qualification is determined based on the time it takes to generate the model. If the release time is less than or equal to the first preset release time, the model loading and initialization operations can be completed quickly within the current timeframe. This indicates sufficient resources and good compatibility between the model and the environment, and the model deployment is considered qualified. If the release time is less than or equal to the second preset release time but greater than the first preset release time, the deployment qualification is further determined based on the data volume of the model file. Data volume affects model loading and initialization time, so a more detailed assessment based on data volume is required. If the release time is still long even when the data volume is less than or equal to the preset data volume, other abnormal factors may exist, and the deployment is considered abnormal. If the data volume exceeds the preset data volume, the large data volume will extend the deployment time, and the first preset release time will be adjusted based on the data volume. Considering the impact of data volume on deployment time, we rationally evaluate model deployment. This improves the accuracy of model evaluation and further enhances model generation efficiency.

[0087] Specifically, the data volume will be adjusted to a value corresponding to the first preset release duration, wherein,

[0088] The increase in the first preset publishing time is proportional to the amount of data.

[0089] In this embodiment, optionally,

[0090] Comparing the data volume with a first preset data volume comparison threshold and a second preset data volume comparison threshold;

[0091] If the data volume is less than or equal to the first preset data volume comparison threshold, the first preset publishing duration is adjusted to 1.1 times the initial first preset publishing duration;

[0092] If the data volume is less than or equal to the second preset data volume comparison threshold and greater than the first preset data volume comparison threshold, the first preset publishing duration is adjusted to 1.2 times the initial first preset publishing duration;

[0093] If the data volume is greater than the second preset data volume comparison threshold, the first preset publishing duration is adjusted to 1.28 times the initial first preset publishing duration;

[0094] The first preset data volume comparison threshold is 1.6S0, and the second preset data volume comparison threshold is 2.7S0.

[0095] Specifically, after completing the adjustment for the first preset release time, the model deployment is determined to be qualified based on the release time again. If the release time is less than or equal to the adjusted first preset release time, the model deployment is determined to be qualified, and the API address and parameter template are generated for the deployed model; if the release time is greater than the adjusted first preset release time, the model deployment is determined to be abnormal, and the model deployment parameters are adjusted based on the time deviation.

[0096] Specifically, the process of adjusting the deployment parameters of the model based on the duration deviation includes:

[0097] The difference between the calculated release duration and the second preset release duration is recorded as the duration deviation;

[0098] If the duration deviation is less than or equal to the first preset duration deviation, adjusting the allocated memory of the Docker container to a corresponding value based on the model generation frequency of the server;

[0099] If the duration deviation is less than or equal to the second preset duration deviation and greater than the first preset duration deviation, adjusting the TCP window to a corresponding value based on the current data transmission rate;

[0100] If the duration deviation is greater than the second predetermined duration deviation, the compatibility detection period used for compatibility checking is adjusted to a corresponding value based on the abnormal frequency.

[0101] Specifically, the model generation frequency of the server is the ratio of the number of new models generated within a preset monitoring period to the preset monitoring period;

[0102] Specifically, the abnormality frequency is the ratio of the number of deployment abnormalities determined as models within a preset assessment period to the preset assessment period.

[0103] Specifically, the current data transmission rate can be calculated by using the psutil library to obtain the traffic information of the network interface.

[0104] Specifically, the first preset time length deviation is 1.3T2, and the second preset time length deviation is 2.7T2.

[0105] Specifically, through research on the relationship between server resource usage and model deployment time, we found that adjusting memory allocation can effectively improve deployment when the deviation is within 1.3T2. Extensive network performance testing and model deployment data analysis revealed that when the deviation is within the range of 1.3T2-2.7T2, network transmission issues are the primary cause of deployment delays, and adjusting the TCP window can effectively improve this.

[0106] Specifically, the allocated memory of the Docker container is adjusted to the corresponding value based on the model generation frequency of the server, where

[0107] The amount of allocated memory increases proportionally to the frequency of model generation.

[0108] In this embodiment, optionally,

[0109] comparing the model generation frequency with the first preset generation frequency and the second preset generation frequency;

[0110] If the model generation frequency is less than or equal to the first preset generation frequency, the allocated memory of the Docker container is adjusted to 1.11 times the initial allocated memory;

[0111] If the model generation frequency is less than or equal to the second preset generation frequency and greater than the first preset generation frequency, the allocated memory of the Docker container is adjusted to 1.23 times the initial allocated memory;

[0112] If the model generation frequency is greater than the second preset generation frequency, the allocated memory of the Docker container is adjusted to 1.31 times the initial allocated memory;

[0113] The preset monitoring period is 3600s, the first preset generation frequency is 0.01, and the second preset generation frequency is 0.05.

[0114] Specifically, the TCP window is adjusted to a corresponding value based on the current data transmission rate, where

[0115] The increase in the TCP window is inversely proportional to the data transmission rate.

[0116] In this embodiment, optionally,

[0117] comparing the data transmission rate with a first preset transmission rate and a second preset transmission rate;

[0118] If the data transmission rate is less than or equal to the first preset transmission rate, the TCP window is adjusted to 1.28 times the initial TCP window;

[0119] If the data transmission rate is less than or equal to the second preset transmission rate and greater than the first preset transmission rate, the TCP window is adjusted to 1.21 times the initial TCP window;

[0120] If the data transmission rate is greater than the second predetermined transmission rate, the TCP window is adjusted to 1.13 times the initial TCP window;

[0121] The first preset transmission rate is 1 MB / s, and the second preset transmission rate is 0.015 MB / s.

[0122] In this embodiment, optionally, the initial TCP window is 1024.

[0123] Specifically, when the model is deployed on TensorFlow Serving, the TCP window that is adjusted is the TCP window of the network socket on the TensorFlow Serving server. When a client sends a request to the TensorFlow Serving server and the server returns a response to the client, data is transmitted based on the TCP protocol. Each TCP connection has a send window and a receive window, which control the amount of data the sender can send and the amount of data the receiver can receive, respectively. The TCP window adjusted in this solution is the window size of the corresponding TCP connection when the server processes the client request.

[0124] After the TCP window is adjusted, the compression ratio of the model file is adjusted to the corresponding value based on the difference between the adjusted TCP window and the initial TCP window, where

[0125] The difference between the adjusted TCP window and the initial TCP window is recorded as the window difference;

[0126] The compression ratio increases in direct proportion to the window difference.

[0127] In this embodiment, optionally,

[0128] Comparing the window difference with a first preset window difference and a second preset window difference;

[0129] If the window difference is less than or equal to the first preset window difference, the compression ratio of the model file is adjusted to 1.13 times the initial compression ratio;

[0130] If the window difference is less than or equal to the second preset window difference and greater than the first preset window difference, the compression ratio of the model file is adjusted to 1.21 times the initial compression ratio;

[0131] If the window difference is greater than the second preset window difference, the compression ratio of the model file is adjusted to 1.29 times the initial compression ratio;

[0132] The first preset window difference is 200B, and the second preset window difference is 300.

[0133] Specifically, the compression ratio is the ratio of the original model file size to the compressed model file size.

[0134] Specifically, the compatibility detection period for compatibility checking is adjusted to a corresponding value based on the abnormal frequency, wherein,

[0135] The reduction in the compatibility detection period is proportional to the anomaly frequency.

[0136] In this embodiment, optionally,

[0137] Compare the abnormal frequency with the first preset abnormal frequency and the second preset abnormal frequency,

[0138] If the abnormal frequency is less than or equal to the first preset abnormal frequency, the compatibility detection period is adjusted to 0.92 times the initial compatibility detection period;

[0139] If the abnormal frequency is less than or equal to the second preset abnormal frequency and greater than the first preset abnormal frequency, the compatibility detection period is adjusted to 0.84 times the initial compatibility detection period;

[0140] If the abnormal frequency is greater than the second preset abnormal frequency, the compatibility detection period is adjusted to 0.72 times the initial compatibility detection period;

[0141] The preset assessment period is 86400s, the first preset abnormal frequency is 0.007, and the second preset abnormal frequency is 0.012.

[0142] Specifically, when the adjustment for the compatibility detection cycle is completed, the compatibility is optimized, including using a compatibility layer, and the compatibility layer is used to convert the model.

[0143] Specifically, when the adjustment for the compatibility detection period is completed, the release duration is re-acquired, and when the re-acquired release duration is greater than the first preset release duration, the configuration file of TensorFlow Serving is updated.

[0144] Specifically, updating the TensorFlow Serving configuration file includes modifying the configuration path and restarting the TensorFlow Serving service after modifying the configuration file to make the new configuration take effect.

[0145] Specifically, model deployment parameters are adjusted based on the duration deviation. The duration deviation reflects the extent to which model deployment exceeds normal time. The severity of the problem is determined based on the size of the duration deviation, and different adjustment strategies are implemented. If the duration deviation is less than or equal to a first preset duration deviation, the model deployment time is prolonged due to insufficient server resources. Therefore, the allocated memory for the Docker container is adjusted based on the server's model generation frequency. The server's model generation frequency indicates how busy the server is. When the model generation frequency is high, the server requires more resources to handle the generation and deployment of new models. By increasing the allocated memory for the Docker container, the server's processing capacity is improved to cope with the high frequency of model generation tasks. If the duration deviation is less than or equal to a second preset duration deviation and greater than the first preset duration deviation, the model deployment time is prolonged due to network transmission issues. This may be caused by network congestion or an inappropriate TCP window setting, resulting in a low data transmission rate. In this case, the TCP window size is adjusted based on the current data transmission rate, which indicates the network's transmission capacity. By adjusting the TCP window size, data transmission efficiency is optimized and the speed of model file transmission is increased. When the duration deviation is greater than the second preset duration deviation, in this case, the model deployment anomaly caused by compatibility issues causes the model release time to be too long. In this case, by reducing the compatibility detection cycle and performing compatibility checks more frequently, compatibility issues can be discovered and resolved in a timely manner, thereby further improving the model generation efficiency.

[0146] Thus far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will fall within the scope of protection of the present invention.

[0147] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that the present invention is susceptible to various modifications and variations. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A dynamic model publishing method based on TensorFlow, characterized in that: include: Deploy TensorFlow Serving; Deploy the model; Generate model configuration and publish the model; The eligibility of a model is determined based on the time it takes to generate the model and the amount of data in the model file, including: When the model deployment is determined to be abnormal, the model deployment parameters are adjusted based on the duration deviation; Or, when the model deployment is determined to be qualified, an API address and parameter template are generated for the deployed model; A process for determining whether the deployment of the model is qualified based on the monitored release duration of the model, including determining that the deployment of the model is abnormal when the release duration is greater than a second preset release duration, and adjusting the deployment parameters of the model based on the duration deviation; When the release duration is less than or equal to the second preset release duration and greater than the first preset release duration, determining whether the model deployment is qualified based on the data volume of the model file; and when the data volume is greater than the preset data volume, adjusting the first preset release duration to a corresponding value based on the data volume, wherein the increase in the first preset release duration is proportional to the data volume; The process of determining whether the deployment of the model is qualified based on the data volume of the model file, further comprising determining that the deployment of the model is abnormal when the data volume is less than or equal to a preset data volume, and adjusting the deployment parameters of the model based on the duration deviation; The process of adjusting the model's deployment parameters based on the duration deviation includes: Determine the difference between the calculated release duration and the second preset release duration as a duration deviation; When the duration deviation is less than or equal to a first preset duration deviation, adjusting the allocated memory of the Docker container to a corresponding value based on the model generation frequency of the server; When the duration deviation is less than or equal to the second preset duration deviation and greater than the first preset duration deviation, adjusting the TCP window to a corresponding value based on the current data transmission rate; When the duration deviation is greater than a second preset duration deviation, the compatibility detection period used for compatibility checking is adjusted to a corresponding value based on the abnormal frequency.

2. The TensorFlow-based model dynamic publishing method according to claim 1, characterized in that: Adjust the allocated memory of the Docker container to the corresponding value based on the model generation frequency of the server, where The amount of allocated memory increases proportionally to the frequency of model generation.

3. The TensorFlow-based model dynamic publishing method according to claim 2, characterized in that: The TCP window is adjusted to the corresponding value based on the current data transmission rate, where The increase in the TCP window is inversely proportional to the data transmission rate.

4. The TensorFlow-based model dynamic publishing method according to claim 3, characterized in that: After the TCP window is adjusted, the compression ratio of the model file is adjusted to the corresponding value based on the difference between the adjusted TCP window and the initial TCP window, where The difference between the adjusted TCP window and the initial TCP window is recorded as the window difference; The compression ratio increases in direct proportion to the window difference.

5. The method for dynamically publishing a model based on TensorFlow according to claim 4, characterized in that: The compatibility detection period for compatibility checking is adjusted to a corresponding value based on the abnormal frequency, wherein, The reduction in the compatibility detection period is proportional to the anomaly frequency.

6. The TensorFlow-based model dynamic publishing method according to claim 5, characterized in that: When the adjustment for the compatibility detection period is completed, the release duration is re-obtained, and when the re-obtained release duration is greater than the first preset release duration, the configuration file of TensorFlow Serving is updated.

Images