A large model product delivery deployment method and system

Abstract

Embodiments of the present application provide a large model product delivery deployment method and system. The method first packs the medium file into an original package file, solves the problem of lack of standardized packaging process and unified medium format specification in the prior art, and determines the original hash value corresponding to the original hash value file for each original package file, which is used to verify the integrity of the data; then, each original package file and the corresponding original hash value file are uploaded to the specified location of the storage bucket, and the large model file in the storage bucket is synchronously encrypted for protection, ensuring the security and integrity of the data in the transmission process, and an exclusive download link with a validity period is generated for each original package file and the corresponding original hash value file, preventing unauthorized access and data leakage; finally, based on the deployment management code in the deployment management directory, the service deployment of the large model product is completed, which can significantly improve the data security of the whole process from research and development to delivery of the large model product.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method and system for delivering and deploying large-scale model products. Background Technology

[0002] With the rapid development of computer technology and artificial intelligence, large-scale model products have emerged, and their application scenarios from R&D to project delivery are becoming increasingly widespread. Therefore, the connection between the R&D and project delivery of large-scale model products has become a core industry requirement.

[0003] However, the delivery and deployment of existing large-scale model products often lack standardized packaging and transmission processes and data integrity and security verification mechanisms. They also lack efficient and secure download link generation mechanisms, which increases the risk of data loss or corruption and further reduces data security. Summary of the Invention

[0004] This invention provides a method and system for delivering and deploying large-scale model products, which can improve data security.

[0005] In a first aspect, embodiments of the present invention provide a method for delivering and deploying a large-scale model product, the method comprising: Based on a preset packaging specification, the media files of a large model product are packaged into at least one original package file, wherein the media files include: deployment management code, environment configuration, model weights, and algorithm scheduling service code; Obtain the original hash value of each of the original packaged files, generate a corresponding original hash value file, and upload each of the original packaged files and the corresponding original hash value file to the target location of the storage bucket; The large model files in the storage bucket are encrypted synchronously, and a unique download link with an expiration date is generated for each original packaged file and the corresponding original hash value file. The large model product is deployed using the deployment management directory where the deployment management code is located.

[0006] Preferably, After synchronously encrypting the large model files in the storage bucket and generating a unique download link with an expiration date for each original packaged file and its corresponding original hash value file, before deploying the large model product service using the deployment management directory where the deployment management code is located, the process further includes: Download the currently packaged file and its corresponding current hash value file using the specified download link; Based on each original packaged file and its corresponding original hash value file, determine whether the values ​​of the current packaged file and the original hash value file are consistent. When it is determined that the values ​​of the current hash value file and the original hash value file are consistent, the service deployment of the large model product is performed using the deployment management directory where the deployment management code is located.

[0007] Preferably, The deployment of the large model product service using the deployment management directory where the deployment management code is located includes: Using the deployment scripts and deployment documentation in the deployment management directory, the current installation status of the basic middleware of the large model product is detected, wherein the current installation status includes: not installed, installed but not running, and installed and running; When the current installation status is detected as not installed, the operating system information of the target server is detected, wherein the operating system information includes: operating system type and operating system version; Obtain the basic middleware from the target source download address in the deployment management directory and install it; Based on the preset configuration file and user parameters, the basic middleware parameters are configured, including: environment variable parameters, service port parameters, and performance parameters. A startup operation is performed on the configured basic middleware, and the running status of the configured basic middleware is verified using the first preset test command in the deployment script.

[0008] Preferably, After performing a startup operation on the configured basic middleware and verifying the running status of the configured basic middleware using the first preset test command in the deployment script, the process further includes: The system receives the input target large model parameters and uses the model startup script in the deployment management directory to read the parameter configuration file of the target large model parameters, wherein the target large model parameters include: model name, version number, and graphics card name; The model startup script is used to detect the current service status of the target large model, and if the current service status is detected as not running, the available graphics card resources are detected. Based on the model name and the version number, the corresponding parameter configuration is read from the parameter configuration file and the loading and inference operation of the target large model is started. The parameter configuration includes: model path and inference parameters. The loading status of the target large model is polled according to a preset period until the target large model is loaded and in a reasoning-ready state; A test request is sent to the target large model service using a second preset test command to verify the running status of the target large model.

[0009] Preferably, After sending a test request to the target large model service using the second preset test command to verify the running status of the target large model, the method further includes: Using the deployment instructions for the algorithm scheduling service in the deployment management directory, determine whether each dependent service of the algorithm scheduling service is installed and running. When each of the dependent services of the algorithm scheduling service is in the installed and running state, the environment media file in the downloaded current package file is decompressed to the target environment directory; Update the global configuration file and start the algorithm scheduling service based on the deployment instructions document. The global configuration file is used to store variable configurations and the environment configuration. A test request is sent to the algorithm scheduling service using a third preset test command to verify the running status of the algorithm scheduling service; When it is necessary to stop the algorithm scheduling service, the corresponding stop operation shall be performed based on the deployment instructions of the algorithm scheduling service. When it is necessary to view the log files of the algorithm scheduling service, perform the corresponding log viewing operation based on the deployment instructions of the algorithm scheduling service.

[0010] Secondly, embodiments of the present invention provide a delivery and deployment system for large-scale model products, the system comprising: Packaging module: Used to package the media files of a large model product into at least one original package file based on a preset packaging specification, wherein the media files include: deployment management code, environment configuration, model weights, and algorithm scheduling service code; Upload module: used to obtain the original hash value of each of the original packaged files, generate the corresponding original hash value file, and upload each of the original packaged files and the corresponding original hash value file to the target location of the storage bucket; Encryption module: used to synchronously encrypt large model files in the storage bucket, and generate a unique download link with an expiration date for each original packaged file and the corresponding original hash value file; Deployment module: Used to deploy the services of the large model product using the deployment management directory where the deployment management code is located.

[0011] Preferably, Following the encryption module and preceding the deployment module, a verification module is further included; The verification module is used to perform: Download the currently packaged file and its corresponding current hash value file using the specified download link; Based on each original packaged file and its corresponding original hash value file, determine whether the values ​​of the current packaged file and the original hash value file are consistent. When it is determined that the values ​​of the current hash value file and the original hash value file are consistent, the service deployment of the large model product is performed using the deployment management directory where the deployment management code is located.

[0012] Preferably, The deployment module includes: a first deployment unit; The first deployment unit is used to execute: Using the deployment scripts and deployment documentation in the deployment management directory, the current installation status of the basic middleware of the large model product is detected, wherein the current installation status includes: not installed, installed but not running, and installed and running; When the current installation status is detected as not installed, the operating system information of the target server is detected, wherein the operating system information includes: operating system type and operating system version; Obtain the basic middleware from the target source download address in the deployment management directory and install it; Based on the preset configuration file and user parameters, the basic middleware parameters are configured, including: environment variable parameters, service port parameters, and performance parameters. A startup operation is performed on the configured basic middleware, and the running status of the configured basic middleware is verified using the first preset test command in the deployment script.

[0013] Preferably, Following the first deployment unit, a second deployment unit is further included; The second deployment unit is used to perform: The system receives the input target large model parameters and uses the model startup script in the deployment management directory to read the parameter configuration file of the target large model parameters, wherein the target large model parameters include: model name, version number, and graphics card name; The model startup script is used to detect the current service status of the target large model, and if the current service status is detected as not running, the available graphics card resources are detected. Based on the model name and the version number, the corresponding parameter configuration is read from the parameter configuration file and the loading and inference operation of the target large model is started. The parameter configuration includes: model path and inference parameters. The loading status of the target large model is polled according to a preset period until the target large model is loaded and in a reasoning-ready state; A test request is sent to the target large model service using a second preset test command to verify the running status of the target large model.

[0014] Preferably, Following the second deployment unit, a third deployment unit is further included; The third deployment unit is used to perform: Using the deployment instructions for the algorithm scheduling service in the deployment management directory, determine whether each dependent service of the algorithm scheduling service is installed and running. When each of the dependent services of the algorithm scheduling service is in the installed and running state, the environment media file in the downloaded current package file is decompressed to the target environment directory; Update the global configuration file and start the algorithm scheduling service based on the deployment instructions document. The global configuration file is used to store variable configurations and the environment configuration. A test request is sent to the algorithm scheduling service using a third preset test command to verify the running status of the algorithm scheduling service; When it is necessary to stop the algorithm scheduling service, the corresponding stop operation shall be performed based on the deployment instructions of the algorithm scheduling service. When it is necessary to view the log files of the algorithm scheduling service, perform the corresponding log viewing operation based on the deployment instructions of the algorithm scheduling service.

[0015] This invention provides a method and system for delivering and deploying large model products. The method first packages the deployment management code, environment configuration, model weights, algorithm scheduling service code, and other media files output by the developer into a single raw package file. This solves the problem of the lack of standardized packaging processes and unified media format specifications in existing technologies, reducing the risk of data loss or corruption, and improving system compatibility and portability. A corresponding raw hash value file is generated for each raw package file, which can be used to verify data integrity. Next, each raw package file and its corresponding raw hash value file are uploaded to a designated location in a storage bucket. During the upload process, the large model files within the storage bucket are simultaneously encrypted to ensure data security and integrity during transmission. A valid hash value file is generated for each raw package file and its corresponding raw hash value file. Dedicated download links ensure that they are valid only for a specified time, automatically expiring after that period. This effectively prevents unauthorized access and data leakage, and addresses the lack of efficient and secure download link generation mechanisms in existing technologies. It meets project delivery requirements. During project delivery, the generated dedicated download links are provided to designated personnel who can use these links to download the necessary files to a designated server node. The download process verifies the file's hash value to ensure file integrity and tamper-proofing. Finally, the deployment management directory containing the deployment management code guides project delivery personnel in deploying the large-scale model product, resolving the lack of clear deployment guidance in existing technologies that often leads to difficulties during deployment. This process, through standardized packaging, encrypted transmission, and secure link generation, ensures data security, integrity, and efficient delivery, significantly improving data security throughout the entire process of large-scale model product development and delivery. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of a delivery and deployment method for a large-scale model product provided in an embodiment of the present invention; Figure 2 This is a flowchart of another method for delivering and deploying a large-scale model product according to an embodiment of the present invention; Figure 3 This is a schematic diagram of a delivery and deployment system for a large-scale product according to an embodiment of the present invention; Figure 4This is a schematic diagram of another large-scale product delivery and deployment system provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of another delivery and deployment system for a large-scale product provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of another delivery and deployment system for a large-scale product provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of a delivery and deployment system for a large-scale product provided in an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0019] like Figure 1 As shown, this embodiment of the invention provides a method for delivering and deploying a large-scale model product, which includes the following steps: Step 101: Based on the preset packaging specifications, package the media files of the large model product into at least one original package file, wherein the media files include: deployment management code, environment configuration, model weights, and algorithm scheduling service code; Step 102: Obtain the original hash value of each original packaged file, generate the corresponding original hash value file, and upload each original packaged file and the corresponding original hash value file to the target location of the storage bucket; Step 103: Synchronize the large model files in the encrypted storage bucket, and generate a unique download link with an expiration date for each original packaged file and its corresponding original hash value file; Step 104: Deploy services for large-scale products using the deployment management directory where the deployment management code is located.

[0020] This invention provides a method for delivering and deploying large model products. First, the method packages the deployment management code, environment configuration, model weights, algorithm scheduling service code, and other media files output by the developer into a single raw package file. This solves the problem of the lack of standardized packaging processes and unified media format specifications in existing technologies, reducing the risk of data loss or corruption, and improving system compatibility and portability. A corresponding raw hash value file is generated for each raw package file to verify data integrity. Next, each raw package file and its corresponding raw hash value file are uploaded to a designated location in a storage bucket. During the upload process, the large model files within the storage bucket are simultaneously encrypted to ensure data security and integrity during transmission. A unique download link with an expiration date is generated for each raw package file and its corresponding raw hash value file. The secure download link ensures that it is valid only for a specified time and automatically expires after that time, effectively preventing unauthorized access and data leakage. It also addresses the lack of efficient and secure download link generation mechanisms in existing technologies, meeting project delivery requirements. During project delivery, the generated secure download link is provided to designated personnel (e.g., the project manager), who can use it to download the required files to a designated server node and verify the file's hash value during the download process to ensure file integrity and tamper-proofing. Finally, the deployment management directory containing the deployment management code guides project delivery personnel in deploying the large-scale model product, resolving the lack of clear deployment guidance in existing technologies that leads to difficulties for project delivery personnel during deployment. This process, through standardized packaging, encrypted transmission, and secure link generation, ensures data security, integrity, and efficient delivery, significantly improving data security throughout the entire process of large-scale model product development and delivery.

[0021] Specifically, media files (such as GitLab code, conda environment, model weights, and deploy-manager) can be uniformly packaged into tar.gz format. To ensure data security, AWS S3's server-side encryption feature can be used to encrypt the data in the bucket (i.e., Amazon S3 bucket), and AWS KMS can be used to generate encryption keys, thereby providing additional security for the data. Furthermore, AWS S3's pre-signed URL feature can be used to generate encrypted, dedicated download links (e.g., HTTPS links) for the files in the bucket.

[0022] For example, GitLab code can be packaged using the `git archive` command and its hash value calculated using the `sha256sum` command; Conda environments can be packaged using the `conda pack` command and their hash values ​​calculated using the `sha256sum` command; model weights can be packaged using the `tar` command; deployment management code (deploy-manager) can be packaged using the `tar` command; and encrypted HTTPS links can be generated using AWS S3's pre-signed URL feature.

[0023] To verify the integrity of the data, in one embodiment of the present invention, the above embodiment further includes the following steps after step 103 and before step 104: Download the currently packaged file and its corresponding current hash value file using the specified download link; Based on each original packaged file and its corresponding original hash value file, determine whether the values ​​of the current packaged file and the original hash value file are consistent. When it is determined that the values ​​of the current hash value file and the original hash value file are consistent, the service deployment of the large model product is performed using the deployment management directory where the deployment management code is located.

[0024] In this embodiment of the invention, to verify data integrity, hash value verification can be performed. First, the current packaged file and its corresponding hash value file are downloaded via a dedicated download link. Then, the original hash value file generated from the uploaded original packaged file is retrieved from the storage bucket. Next, the value in the current hash value file is compared with the value in the original hash value file. Only when the two values ​​match is the large model product service deployed based on the directory where the deployment management code is located. This process effectively verifies whether the downloaded packaged file is complete and has not been tampered with, further improving data security.

[0025] To ensure the smooth deployment and normal operation of the basic middleware, in one embodiment of the present invention, step 104 in the above embodiment includes: Using the deployment scripts and deployment documentation in the deployment management directory, the current installation status of the basic middleware of the large model product is detected, wherein the current installation status includes: not installed, installed but not running, and installed and running; When the current installation status is detected as not installed, the operating system information of the target server is detected, wherein the operating system information includes: operating system type and operating system version; Obtain the basic middleware from the target source download address in the deployment management directory and install it; Based on the preset configuration file and user parameters, the basic middleware parameters are configured, including: environment variable parameters, service port parameters, and performance parameters. A startup operation is performed on the configured basic middleware, and the running status of the configured basic middleware is verified using the first preset test command in the deployment script.

[0026] In this embodiment of the invention, since the deployment management directory provides deployment scripts and detailed deployment documentation for the basic middleware components, and the deployment documentation clearly explains how to run the automated scripts and the specific meanings of the script parameters, the basic middleware can be deployed through the deployment management directory. Based on this, the automated deployment script can first detect the current installation status of the basic middleware. If the current installation status is detected as not installed, the script automatically identifies the target server's operating system type and version to ensure compatibility with subsequent operations. Then, the automated deployment script downloads and installs the required basic middleware from the target source download address to ensure a smooth installation process. Next, based on the preset configuration file and user parameters, necessary configuration adjustments are made to the basic middleware (e.g., environment variable settings, service port configuration, performance parameter optimization, etc.). Subsequently, the middleware service is started and its normal operation is ensured. A series of first preset test commands are then used to verify the middleware's running status to ensure service stability and availability. The above process ensures that the basic middleware can be successfully deployed and run normally, providing a solid foundation for subsequent large-scale model deployment and operation.

[0027] Specifically, the deployment management directory can be named deploy-manager / dependent_service, the basic middleware can be components such as JDK, Redis, MariaDB, MinIO, and Elasticsearch, and the deployment documentation can be named readme.md.

[0028] Additionally, if the current installation status is detected as "installed but not running," the step of "obtaining and installing the basic middleware from the target source address in the deployment management directory" in the above process is unnecessary; the other processes can be executed normally in sequence. If the current installation status is detected as "installed and running," the steps of "obtaining and installing the basic middleware from the target source address in the deployment management directory" and "performing a startup operation for the configured basic middleware" in the above process are unnecessary; the other processes can be executed normally in sequence.

[0029] To ensure efficient loading and stable operation of large models, in one embodiment of the present invention, after the steps of performing a startup operation on the configured basic middleware and verifying the running status of the configured basic middleware using the first preset test command in the deployment script, the embodiment further includes: The system receives the input target large model parameters and uses the model startup script in the deployment management directory to read the parameter configuration file of the target large model parameters, wherein the target large model parameters include: model name, version number, and graphics card name; The model startup script is used to detect the current service status of the target large model, and if the current service status is detected as not running, the available graphics card resources are detected. Based on the model name and the version number, the corresponding parameter configuration is read from the parameter configuration file and the loading and inference operation of the target large model is started. The parameter configuration includes: model path and inference parameters. The loading status of the target large model is polled according to a preset period until the target large model is loaded and in a reasoning-ready state; A test request is sent to the target large model service using a second preset test command to verify the running status of the target large model.

[0030] In this embodiment of the invention, a parameter configuration file and a model startup script are provided in the deployment management directory, thus enabling the deployment of a target large model service. By passing in target large model parameters such as model name, version number, and graphics card name, the model startup script automatically reads the corresponding parameter configuration file and loads the target large model. Based on this, the current service status of the target large model can be detected first. If the current service status is detected as not running, subsequent operations are performed. If the target large model service is detected as already existing, the process is terminated to ensure that the model can be reloaded without conflict. Next, the model startup script detects available graphics card resources on the server to ensure that the model can efficiently utilize hardware resources for inference. Then, based on the passed model name and version number, the corresponding parameter configuration (e.g., model path, inference parameters, etc.) in the model startup parameter configuration file is read. Subsequently, the model loading process is started using the read configuration parameters to ensure that the model can be correctly loaded onto the specified graphics card. The model loading status is then polled until the model is fully loaded and in an inference-ready state. Finally, a second preset test command is used to send a test request to the model service to verify whether the model has been successfully loaded and can respond normally to inference requests. Through the above process, the model startup script ensures the efficient loading and stable operation of large models, providing project delivery personnel with a simple and reliable way to start and verify model services.

[0031] Specifically, the model startup parameter configuration file can be named model_config.yaml, and the model startup script can be named model_start.py.

[0032] To ensure the smooth startup and stable operation of the algorithm scheduling service, in one embodiment of the present invention, after the step of sending a test request to the target large model service using a second preset test command to verify the running status of the target large model, the embodiment further includes: Using the deployment instructions for the algorithm scheduling service in the deployment management directory, determine whether each dependent service of the algorithm scheduling service is installed and running. When each of the dependent services of the algorithm scheduling service is in the installed and running state, the environment media file in the downloaded current package file is decompressed to the target environment directory; Update the global configuration file and start the algorithm scheduling service based on the deployment instructions document. The global configuration file is used to store variable configurations and the environment configuration. A test request is sent to the algorithm scheduling service using a third preset test command to verify the running status of the algorithm scheduling service; When it is necessary to stop the algorithm scheduling service, the corresponding stop operation shall be performed based on the deployment instructions of the algorithm scheduling service. When it is necessary to view the log files of the algorithm scheduling service, perform the corresponding log viewing operation based on the deployment instructions of the algorithm scheduling service.

[0033] In this embodiment of the invention, after the deployment of the basic middleware service and the large model loading inference service is completed, the algorithm scheduling service needs to be started. The deployment documentation for the algorithm scheduling service provides detailed deployment guidance. To ensure a smooth deployment process, the deployment instructions can be used to perform the deployment operation. Based on this, the running status of each dependent service of the algorithm scheduling service can be determined first. Deployers need to ensure that these services are correctly installed and running on the target server. Then, when each dependent service of the algorithm scheduling service is in the installed and running state, the environment media file (e.g., conda_env.tar.gz) downloaded from the current package file is decompressed to the specified target environment directory (e.g., / data / miniconda3), ensuring the availability of the environment for subsequent activation and use. Next, deployers can complete the startup configuration of the algorithm scheduling service by modifying the global configuration file. The deployment instructions for the algorithm scheduling service detail the configuration decoupling strategy. Fixed configurations are stored internally within the code, while variable configurations and environment configurations are stored in a global configuration file. Subsequently, the algorithm scheduling service can be started according to the specific commands and steps provided in the deployment documentation, ensuring its normal operation. Finally, test requests can be sent to the algorithm scheduling service using the third preset test command outlined in the deployment documentation to verify its operational status and ensure stable service operation. Furthermore, when it is necessary to stop the service or view the algorithm scheduling service's log files, the corresponding operations can be performed based on the deployment documentation, allowing deployment personnel to monitor the service's operational status through log files and promptly identify and resolve issues. Through this process, the algorithm scheduling service deployment documentation provides deployment personnel with a clear and efficient deployment process, ensuring the smooth startup and stable operation of the algorithm scheduling service. Through configuration decoupling and standardized document management, the maintainability and deployability of the algorithm scheduling service are also ensured.

[0034] Specifically, dependent services may include: Miniconda, Redis, Elasticsearch, MariaDB, etc.

[0035] like Figure 2 As shown, to more clearly illustrate the technical solution and advantages of the present invention, the following provides a detailed description of a method for delivering and deploying a large-scale model product, which may include the following steps: Step 201: Based on the preset packaging specifications, package the media files of the large model product into at least one original package file, wherein the media files include: deployment management code, environment configuration, model weights, and algorithm scheduling service code; Step 202: Obtain the original hash value of each original packaged file, generate the corresponding original hash value file, and upload each original packaged file and the corresponding original hash value file to the target location of the storage bucket; Step 203: Synchronize the large model files in the encrypted storage bucket, and generate a unique download link with an expiration date for each original packaged file and its corresponding original hash value file; Step 204: Download the current packaged file and its corresponding current hash value file using the dedicated download link; Step 205: Based on each original packaged file and its corresponding original hash value file, determine whether the values ​​of the current packaged file and the original hash value file are consistent; Step 206: When it is determined that the values ​​of the current hash value file and the original hash value file of the current packaged file are consistent, the current installation status of the basic middleware of the large model product is detected by using the deployment script and deployment instructions in the deployment management directory. The current installation status includes: not installed, installed but not running, and installed and running. Step 207: When the current installation status is detected as not installed, detect the operating system information of the target server, including: operating system type and operating system version; Step 208: Obtain the basic middleware from the target source download address in the deployment management directory and install it; Step 209: Configure the basic middleware parameters according to the preset configuration file and user parameters. The basic middleware parameters include: environment variable parameters, service port parameters, and performance parameters. Step 210: Perform a startup operation on the configured basic middleware and verify the running status of the configured basic middleware using the first preset test command in the deployment script; Step 211: Receive the input target large model parameters and use the model startup script in the deployment management directory to read the parameter configuration file of the target large model parameters. The target large model parameters include: model name, version number, and graphics card name. Step 212: Use the model startup script to detect the current service status of the target large model, and if the current service status is detected as not running, detect the available graphics card resources; Step 213: Based on the model name and version number, read the corresponding parameter configuration from the parameter configuration file and start the loading and inference operation of the target large model. The parameter configuration includes: model path and inference parameters. Step 214: Poll the loading status of the target large model at a preset period until the target large model is loaded and in a reasoning-ready state; Step 215: Use the second preset test command to send a test request to the target large model service to verify the running status of the target large model; Step 216: Use the deployment instructions for the algorithm scheduling service in the deployment management directory to determine whether each dependent service of the algorithm scheduling service is installed and running. Step 217: When all dependent services of the algorithm scheduling service are installed and running, decompress the environment media files in the downloaded current package file to the target environment directory; Step 218: Update the global configuration file and start the algorithm scheduling service based on the algorithm scheduling service deployment instructions document. The global configuration file is used to store variable configurations and environment configurations. Step 219: Send a test request to the algorithm scheduling service using the third preset test command to verify the running status of the algorithm scheduling service; Step 220: When it is necessary to stop the algorithm scheduling service, perform the corresponding stop operation based on the algorithm scheduling service deployment documentation; Step 221: When you need to view the log files of the algorithm scheduling service, perform the corresponding log viewing operation based on the algorithm scheduling service deployment instructions.

[0036] like Figure 3 As shown, this embodiment of the invention provides a delivery and deployment system for large-scale model products, the system comprising: Packaging module 301: Used to package the media files of a large model product into at least one original package file based on a preset packaging specification, wherein the media files include: deployment management code, environment configuration, model weights, and algorithm scheduling service code; Upload module 302: used to obtain the original hash value of each of the original packaged files, generate the corresponding original hash value file, and upload each of the original packaged files and the corresponding original hash value file to the target location of the storage bucket; Encryption module 303: used to synchronously encrypt large model files in the storage bucket, and generate a unique download link with an expiration date for each original packaged file and the corresponding original hash value file; Deployment module 304: Used to deploy the services of the large model product using the deployment management directory where the deployment management code is located.

[0037] based on Figure 3 The example shown is a delivery and deployment system for a large-scale product, such as... Figure 4 As shown, after the encryption module 303 and before the deployment module 304, a verification module 305 is further included; The verification module 305 is used to perform: Download the currently packaged file and its corresponding current hash value file using the specified download link; Based on each original packaged file and its corresponding original hash value file, determine whether the values ​​of the current packaged file and the original hash value file are consistent. When it is determined that the values ​​of the current hash value file and the original hash value file are consistent, the service deployment of the large model product is performed using the deployment management directory where the deployment management code is located.

[0038] based on Figure 4 The example shown is a delivery and deployment system for a large-scale product, such as... Figure 5 As shown, the deployment module 304 includes: a first deployment unit 3041; The first deployment unit 3041 is used to perform: Using the deployment scripts and deployment documentation in the deployment management directory, the current installation status of the basic middleware of the large model product is detected, wherein the current installation status includes: not installed, installed but not running, and installed and running; When the current installation status is detected as not installed, the operating system information of the target server is detected, wherein the operating system information includes: operating system type and operating system version; Obtain the basic middleware from the target source download address in the deployment management directory and install it; Based on the preset configuration file and user parameters, the basic middleware parameters are configured, including: environment variable parameters, service port parameters, and performance parameters. A startup operation is performed on the configured basic middleware, and the running status of the configured basic middleware is verified using the first preset test command in the deployment script.

[0039] based on Figure 5 The example shown is a delivery and deployment system for a large-scale product, such as... Figure 6 As shown, after the first deployment unit 3041, a second deployment unit 3042 is further included; The second deployment unit is used to perform: The system receives the input target large model parameters and uses the model startup script in the deployment management directory to read the parameter configuration file of the target large model parameters, wherein the target large model parameters include: model name, version number, and graphics card name; The model startup script is used to detect the current service status of the target large model, and if the current service status is detected as not running, the available graphics card resources are detected. Based on the model name and the version number, the corresponding parameter configuration is read from the parameter configuration file and the loading and inference operation of the target large model is started. The parameter configuration includes: model path and inference parameters. The loading status of the target large model is polled according to a preset period until the target large model is loaded and in a reasoning-ready state; A test request is sent to the target large model service using a second preset test command to verify the running status of the target large model.

[0040] based on Figure 6 The example shown is a delivery and deployment system for a large-scale product, such as... Figure 7 As shown, after the second deployment unit 3042, a third deployment unit 3043 is further included; The third deployment unit 3043 is used to perform: Using the deployment instructions for the algorithm scheduling service in the deployment management directory, determine whether each dependent service of the algorithm scheduling service is installed and running. When each of the dependent services of the algorithm scheduling service is in the installed and running state, the environment media file in the downloaded current package file is decompressed to the target environment directory; Update the global configuration file and start the algorithm scheduling service based on the deployment instructions document. The global configuration file is used to store variable configurations and the environment configuration. A test request is sent to the algorithm scheduling service using a third preset test command to verify the running status of the algorithm scheduling service; When it is necessary to stop the algorithm scheduling service, the corresponding stop operation shall be performed based on the deployment instructions of the algorithm scheduling service. When it is necessary to view the log files of the algorithm scheduling service, perform the corresponding log viewing operation based on the deployment instructions of the algorithm scheduling service.

[0041] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on a delivery and deployment system for a large-scale product. In other embodiments of the present invention, a delivery and deployment system for a large-scale product may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0042] The information interaction and execution process between the various units in the above-mentioned device are based on the same concept as the method embodiment of the present invention, and the specific details can be found in the description of the method embodiment of the present invention, and will not be repeated here.

[0043] This invention also provides a delivery and deployment system for large-scale products, comprising: at least one memory and at least one processor; At least one memory for storing machine-readable programs; At least one processor is configured to invoke a machine-readable program to execute a delivery and deployment method for a large-scale product according to any embodiment of the present invention.

[0044] This invention also provides a computer-readable medium storing computer instructions, which, when executed by a processor, cause the processor to perform a delivery and deployment method for a large-scale product according to any embodiment of this invention.

[0045] Specifically, a system or apparatus equipped with a storage medium may be provided, on which software program code implementing the functions of any of the embodiments described above is stored, and the computer (or CPU or MPU) of the system or apparatus may read and execute the program code stored in the storage medium.

[0046] In this case, the program code read from the storage medium can itself implement the function of any of the above embodiments, and therefore the program code and the storage medium storing the program code constitute part of the present invention.

[0047] Storage media embodiments for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer via a communication network.

[0048] Furthermore, it should be clear that not only can the program code read by the computer be executed, but also the operating system or other components operating on the computer can be instructed based on the program code to perform some or all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0049] Furthermore, it is understood that the program code read from the storage medium is written to the memory set in the expansion board inserted into the computer or to the memory set in the expansion unit connected to the computer. Then, based on the instructions of the program code, the CPU or other components installed on the expansion board or expansion unit execute some and all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0050] The various embodiments of the present invention have at least the following beneficial effects: 1. In this embodiment of the invention, a method for delivering and deploying a large model product is provided. This method first packages the deployment management code, environment configuration, model weights, algorithm scheduling service code, and other media files output by the R&D team into a unified original package file. This solves the problem of the lack of standardized packaging processes and unified media format specifications in existing technologies, reduces the risk of data loss or corruption, and improves system compatibility and portability. A corresponding original hash value file is generated for each original package file, which can be used to verify data integrity. Next, each original package file and its corresponding original hash value file are uploaded to a designated location in a storage bucket. During the upload process, the large model files in the storage bucket are simultaneously encrypted to ensure the security and integrity of the data during transmission. A valid hash value file is generated for each original package file and its corresponding original hash value file. Dedicated download links ensure that they are valid only for a specified time, automatically expiring after that period. This effectively prevents unauthorized access and data leakage, and addresses the lack of efficient and secure download link generation mechanisms in existing technologies. It meets project delivery requirements. During project delivery, the generated dedicated download links are provided to designated personnel who can use these links to download the necessary files to a designated server node. The download process verifies the file's hash value to ensure file integrity and tamper-proofing. Finally, the deployment management directory containing the deployment management code guides project delivery personnel in deploying the large-scale model product, resolving the lack of clear deployment guidance in existing technologies that leads to difficulties during deployment. This process, through standardized packaging, encrypted transmission, and secure link generation, ensures data security, integrity, and efficient delivery, significantly improving data security throughout the entire process of large-scale model product development and delivery. 2. In this embodiment of the invention, to verify data integrity, hash value verification can be performed. First, the current packaged file and its corresponding hash value file need to be downloaded via a dedicated download link; then, the original hash value file generated from the uploaded original packaged file is obtained from the storage bucket; next, the value of the current hash value file is compared with the value of the original hash value file, and only when the two values ​​are confirmed to be consistent is the service deployment of the large model product based on the directory where the deployment management code is located. The above process can effectively verify whether the downloaded current packaged file is complete and has not been tampered with, further improving data security. 3. In this embodiment of the invention, since the deployment management directory provides deployment scripts and detailed deployment instructions for the basic middleware components, and the deployment instructions clearly explain how to run the automated scripts and the specific meanings of the script parameters, the basic middleware can be deployed through the deployment management directory. Based on this, the current status of the basic middleware can be detected first using the automated deployment script. If the current installation status is detected as not installed, the operating system type and version of the target server are automatically identified to ensure compatibility with subsequent operations. Then, the required basic middleware is downloaded and installed from the target source download address using the automated deployment script to ensure a smooth installation process. Next, necessary configuration adjustments are made to the basic middleware according to the preset configuration file and user parameters. Subsequently, the middleware service is started and its normal operation is ensured. A series of first preset test commands are used to verify the middleware's running status to ensure service stability and availability. The above process ensures that the basic middleware can be successfully deployed and run normally, providing a solid foundation for the subsequent deployment and operation of large-scale models.

[0051] In the above embodiments, the hardware units can be implemented mechanically or electrically. For example, a hardware unit may include permanent dedicated circuitry or logic (such as a dedicated processor, FPGA, or ASIC) to perform the corresponding operation. The hardware unit may also include programmable logic or circuitry (such as a general-purpose processor or other programmable processor), which can be temporarily configured by software to perform the corresponding operation. The specific implementation method (mechanical, dedicated permanent circuitry, or temporarily configured circuitry) can be determined based on cost and time considerations.

[0052] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for delivering and deploying a large-scale model product, characterized in that, The method includes: Based on a preset packaging specification, the media files of a large model product are packaged into at least one original package file, wherein the media files include: deployment management code, environment configuration, model weights, and algorithm scheduling service code; Obtain the original hash value of each of the original packaged files, generate a corresponding original hash value file, and upload each of the original packaged files and the corresponding original hash value file to the target location of the storage bucket; The large model files in the storage bucket are encrypted synchronously, and a unique download link with an expiration date is generated for each original packaged file and the corresponding original hash value file. The large model product is deployed using the deployment management directory where the deployment management code is located.

2. The method according to claim 1, characterized in that, After synchronously encrypting the large model files in the storage bucket and generating a unique download link with an expiration date for each original packaged file and its corresponding original hash value file, before deploying the large model product service using the deployment management directory where the deployment management code is located, the process further includes: Download the currently packaged file and its corresponding current hash value file using the specified download link; Based on each original packaged file and its corresponding original hash value file, determine whether the values ​​of the current packaged file and the original hash value file are consistent. When it is determined that the values ​​of the current hash value file corresponding to the current packaged file and the original hash value file are consistent, the service deployment of the large model product is performed using the deployment management directory where the deployment management code is located.

3. The method according to claim 1, characterized in that, The deployment of the large model product service using the deployment management directory where the deployment management code is located includes: Using the deployment scripts and deployment documentation in the deployment management directory, the current installation status of the basic middleware of the large model product is detected, wherein the current installation status includes: not installed, installed but not running, and installed and running; When the current installation status is detected as not installed, the operating system information of the target server is detected, wherein the operating system information includes: operating system type and operating system version; Obtain the basic middleware from the target source download address in the deployment management directory and install it; Based on the preset configuration file and user parameters, the basic middleware parameters are configured, including: environment variable parameters, service port parameters, and performance parameters. A startup operation is performed on the configured basic middleware, and the running status of the configured basic middleware is verified using the first preset test command in the deployment script.

4. The method according to claim 3, characterized in that, After performing a startup operation on the configured basic middleware and verifying the running status of the configured basic middleware using the first preset test command in the deployment script, the process further includes: The system receives the input target large model parameters and uses the model startup script in the deployment management directory to read the parameter configuration file of the target large model parameters, wherein the target large model parameters include: model name, version number, and graphics card name; The model startup script is used to detect the current service status of the target large model, and if the current service status is detected as not running, the available graphics card resources are detected. Based on the model name and the version number, the corresponding parameter configuration is read from the parameter configuration file and the loading and inference operation of the target large model is started. The parameter configuration includes: model path and inference parameters. The loading status of the target large model is polled according to a preset period until the target large model is loaded and in a reasoning-ready state; A test request is sent to the target large model service using a second preset test command to verify the running status of the target large model.

5. The method according to claim 4, characterized in that, After sending a test request to the target large model service using the second preset test command to verify the running status of the target large model, the method further includes: Using the deployment instructions for the algorithm scheduling service in the deployment management directory, determine whether each dependent service of the algorithm scheduling service is installed and running. When each of the dependent services of the algorithm scheduling service is in the installed and running state, the environment media file in the downloaded current package file is decompressed to the target environment directory; Update the global configuration file and start the algorithm scheduling service based on the deployment instructions document. The global configuration file is used to store variable configurations and the environment configuration. A test request is sent to the algorithm scheduling service using a third preset test command to verify the running status of the algorithm scheduling service; When it is necessary to stop the algorithm scheduling service, the corresponding stop operation shall be performed based on the deployment instructions of the algorithm scheduling service. When it is necessary to view the log files of the algorithm scheduling service, perform the corresponding log viewing operation based on the deployment instructions of the algorithm scheduling service.

6. A delivery and deployment system for large-scale model products, characterized in that, The system includes: Packaging module: Used to package the media files of a large model product into at least one original package file based on a preset packaging specification, wherein the media files include: deployment management code, environment configuration, model weights, and algorithm scheduling service code; Upload module: used to obtain the original hash value of each of the original packaged files, generate the corresponding original hash value file, and upload each of the original packaged files and the corresponding original hash value file to the target location of the storage bucket; Encryption module: used to synchronously encrypt large model files in the storage bucket, and generate a unique download link with an expiration date for each original packaged file and the corresponding original hash value file; Deployment module: Used to deploy the services of the large model product using the deployment management directory where the deployment management code is located.

7. The system according to claim 6, characterized in that, Following the encryption module and preceding the deployment module, a verification module is further included; The verification module is used to perform: Download the currently packaged file and its corresponding current hash value file using the specified download link; Based on each original packaged file and its corresponding original hash value file, determine whether the values ​​of the current packaged file and the original hash value file are consistent. When it is determined that the values ​​of the current hash value file and the original hash value file are consistent, the service deployment of the large model product is performed using the deployment management directory where the deployment management code is located.

8. The system according to claim 7, characterized in that, The deployment module includes: a first deployment unit; The first deployment unit is used to execute: Using the deployment scripts and deployment documentation in the deployment management directory, the current installation status of the basic middleware of the large model product is detected, wherein the current installation status includes: not installed, installed but not running, and installed and running; When the current installation status is detected as not installed, the operating system information of the target server is detected, wherein the operating system information includes: operating system type and operating system version; Obtain the basic middleware from the target source download address in the deployment management directory and install it; Based on the preset configuration file and user parameters, the basic middleware parameters are configured, including: environment variable parameters, service port parameters, and performance parameters. A startup operation is performed on the configured basic middleware, and the running status of the configured basic middleware is verified using the first preset test command in the deployment script.

9. The system according to claim 8, characterized in that, Following the first deployment unit, a second deployment unit is further included; The second deployment unit is used to perform: The system receives the input target large model parameters and uses the model startup script in the deployment management directory to read the parameter configuration file of the target large model parameters, wherein the target large model parameters include: model name, version number, and graphics card name; The model startup script is used to detect the current service status of the target large model, and if the current service status is detected as not running, the available graphics card resources are detected. Based on the model name and the version number, the corresponding parameter configuration is read from the parameter configuration file and the loading and inference operation of the target large model is started. The parameter configuration includes: model path and inference parameters. The loading status of the target large model is polled according to a preset period until the target large model is loaded and in a reasoning-ready state; A test request is sent to the target large model service using a second preset test command to verify the running status of the target large model.

10. The system according to claim 9, characterized in that, Following the second deployment unit, a third deployment unit is further included; The third deployment unit is used to perform: Using the deployment instructions for the algorithm scheduling service in the deployment management directory, determine whether each dependent service of the algorithm scheduling service is installed and running. When each of the dependent services of the algorithm scheduling service is in the installed and running state, the environment media file in the downloaded current package file is decompressed to the target environment directory; Update the global configuration file and start the algorithm scheduling service based on the deployment instructions document. The global configuration file is used to store variable configurations and the environment configuration. A test request is sent to the algorithm scheduling service using a third preset test command to verify the running status of the algorithm scheduling service; When it is necessary to stop the algorithm scheduling service, the corresponding stop operation shall be performed based on the deployment instructions of the algorithm scheduling service. When it is necessary to view the log files of the algorithm scheduling service, perform the corresponding log viewing operation based on the deployment instructions of the algorithm scheduling service.

Images