Mirror file synchronization method and apparatus, storage medium, and electronic device

Abstract

The application provides a synchronization method and device of a mirror file, a storage medium and an electronic device. The method comprises the following steps: in response to a synchronization instruction, synchronizing a mirror file to be synchronized from a first container mirror warehouse to a second container mirror warehouse. The first container mirror warehouse and the second container mirror warehouse belong to the same type of mirror warehouse. The mirror files in the second container mirror warehouse and the mirror files in a target database of the mirror files that can be recognized by a storage inference platform are traversed. According to the traversal result, the mirror files with differences between the second container mirror warehouse and the target database are determined, and target mirror files are obtained. The mirror files in the second container mirror warehouse are synchronized to the target database based on the target mirror files, so that the inference platform uses the mirror files in the target database to provide services to a target object. Through the application, the problem of low synchronization efficiency of the mirror files in the related art is solved, and the synchronization efficiency of the mirror files is improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the computer field, in particular, to a mirror image file synchronization method and device, a storage medium and an electronic device. BACKGROUND

[0002] At present, a mirror image warehouse can be used to store mirror image files, and there can be a need for mirror image synchronization between multiple mirror image warehouses. In related technologies, the following two schemes are usually used to realize synchronization of mirror image files between different mirror image warehouses:

[0003] The first scheme uses the data mirror image backup tool RSYNC provided by Linux to realize synchronization of mirror image files between two mirror image warehouses.

[0004] The second scheme, for a scenario of using IaaS (Infrastructure as a Service, Infrastructure as a Service) service to store mirror images, uses the configuration tool of IaaS to configure synchronization of mirror image files.

[0005] From the above content, it can be seen that the above two methods need to take different tool strategies for the storage environment where the mirror image warehouse is located. In the interaction scenario of the mirror image warehouse of the inference platform and the mirror image warehouse of other platforms or environments, as shown in Figure 1 , it is usually necessary to save the mirror image files of other mirror image warehouses from the server to the local, and upload the locally saved mirror image files to the mirror image warehouse of the inference platform (i.e. Figure 1 mirror image transfer). Then, the mirror image files in the mirror image warehouse of the inference platform are synchronized to the mirror image database of the inference platform, so that the inference platform can read and use the mirror image files from the mirror image database.

[0006] Usually, the mirror image files trained by the training platform can reach 10G or more, so the process of synchronizing the mirror image files will be very long, and the synchronization efficiency of the mirror image files is relatively low. Moreover, in the process of synchronizing the mirror image files to the local or uploading to the inference platform, if any error occurs, it needs to be re-synchronized, which further reduces the synchronization efficiency of the mirror image files. SUMMARY

[0007] Embodiments of the present application provide a mirror image file synchronization method, device, storage medium and electronic device to at least solve the problem of low synchronization efficiency of mirror image files in related technologies.

[0008] According to one embodiment of the present application, a synchronization method of a mirror file is provided, including: in response to a synchronization instruction, synchronizing a mirror file to be synchronized from a first container mirror warehouse to a second container mirror warehouse, wherein the first container mirror warehouse is used to store mirror files generated by a training platform for a preset training scenario, the second container mirror warehouse is used to store mirror files required by an inference platform to provide services for a target object, and the first container mirror warehouse and the second container mirror warehouse belong to the same type of mirror warehouse; traversing the mirror files in the second container mirror warehouse and traversing the mirror files in a target database to obtain a traversal result, wherein the target database is used to store mirror files that can be recognized by the inference platform; determining, according to the traversal result, mirror files that exist in differences between the second container mirror warehouse and the target database to obtain target mirror files; and synchronizing the mirror files in the second container mirror warehouse into the target database based on the target mirror files, so that the inference platform uses the mirror files in the target database to provide services for the target object.

[0009] In one example embodiment, the synchronization method of the mirror file further includes: parsing the synchronization instruction to determine a project identification of a source project, a project identification of a target project, and user information of a target user, wherein the target user is a user who issues the synchronization instruction; determining the first container mirror warehouse according to the project identification of the source project and determining the second container mirror warehouse according to the project identification of the target project; establishing a connection between the first container mirror warehouse and the second container mirror warehouse according to the user information of the target user and determining a synchronization permission of the target user, wherein the synchronization permission represents whether the target user can access the mirror files in the first container mirror warehouse; and synchronizing the mirror file to be synchronized from the first container mirror warehouse to the second container mirror warehouse when the synchronization permission of the target user reaches a target permission.

[0010] In one example embodiment, the synchronization method of the mirror file further includes: after synchronizing the mirror file to be synchronized from the first container mirror warehouse to the second container mirror warehouse, synchronizing the updated mirror file from the first container mirror warehouse to the second container mirror warehouse when the mirror file in the first container mirror warehouse is updated.

[0011] In an example embodiment, the synchronization method of the image file further comprises: obtaining a target file identifier and a target file size of the first image file in the second container image warehouse according to the traversal result; determining the first image file as a first target image file when there is no image file in the target database with a file identifier same as the target file identifier of the first image file and a file size same as the target file size of the first image file; obtaining a target file identifier and a target file size of a second image file in the target database according to the traversal result; and determining the second image file as a second target image file when there is no image file in the second container image warehouse with a file identifier same as the target file identifier of the second image file and a file size same as the target file size of the second image file.

[0012] In an example embodiment, the synchronization method of the image file further comprises: when the target image file is the first target image file, obtaining service information of a service provided by the inference platform, a file identifier of the first target image file, and image synchronization information of the first target image file; generating target data based on the service information, the file identifier of the first target image file, and the image synchronization information; and synchronizing the target data and the first target image file to the target database.

[0013] In an example embodiment, the synchronization method of the image file further comprises: when the target image file is the second target image file, deleting the second target image file from the target database.

[0014] In an example embodiment, the synchronization method of the image file further comprises: when it is detected that the image file in the second container image warehouse has an update, obtaining the updated image file; and synchronizing the updated image file to the first container image warehouse.

[0015] In an example embodiment, the synchronization method of the image file further comprises: after synchronizing the image file in the second container image warehouse to the target database based on the target image file, synchronizing the image file corresponding to the first inference platform to the container image warehouse corresponding to the second inference platform in response to a platform synchronization instruction, wherein the container image warehouse of the first inference platform is the same as the container image warehouse of the second inference platform.

[0016] According to another embodiment of the present application, a synchronization device of a mirror image file is provided, comprising: a first synchronization module, configured to synchronize a mirror image file to be synchronized from a first container image warehouse to a second container image warehouse in response to a synchronization instruction, wherein the first container image warehouse is configured to store a mirror image file generated by a training platform for a preset training scenario, the second container image warehouse is configured to store a mirror image file required by an inference platform to provide a service for a target object, and the first container image warehouse and the second container image warehouse belong to the same type of image warehouse; a traversal module, configured to traverse the mirror image file in the second container image warehouse and traverse a mirror image file in a target database to obtain a traversal result, wherein the target database is configured to store a mirror image file recognizable by the inference platform; a difference determination module, configured to determine a mirror image file with a difference between the second container image warehouse and the target database according to the traversal result to obtain a target mirror image file; and a second synchronization module, configured to synchronize the mirror image file in the second container image warehouse to the target database based on the target mirror image file, so that the inference platform uses the mirror image file in the target database to provide a service for the target object.

[0017] According to still another embodiment of the present application, a computer readable storage medium is also provided, and the computer readable storage medium stores a computer program, wherein the computer program is set to execute the synchronization method of the mirror image file when running.

[0018] According to still another embodiment of the present application, an electronic device is also provided, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is set to execute the computer program to execute the synchronization method of the mirror image file.

[0019] According to the present application, since the container image warehouse has the “copy” feature, in the process of synchronizing the mirror image files of different container image warehouses, the synchronization between the different container image warehouses can be realized directly based on the “copy” feature of the container image warehouse, without the need to save the mirror image file to the local or upload the mirror image file to the inference platform. When the mirror image file is large, the synchronization between the different container image warehouses can also be quickly realized, thereby improving the synchronization efficiency of the mirror image file. In addition, in the present application, in the process of synchronizing the mirror image file in the container image warehouse of the inference platform to the image database of the inference platform, the image database is synchronized based on the difference of the mirror image file between the container image warehouse and the image database, and the whole mirror image file in the container image warehouse is not copied. Therefore, in the process of synchronizing the container image warehouse of the inference platform and the image database of the inference platform, only the mirror image file with the difference needs to be synchronized, thereby further improving the synchronization efficiency of the mirror image file. Moreover, even if any error occurs in the synchronization process, the re-synchronization will not significantly reduce the synchronization efficiency.

[0020] Therefore, the scheme provided in the application can solve the problem of low synchronization efficiency of mirror files in the related art, and achieve the effect of improving the synchronization efficiency of mirror files. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 is a flowchart of a mirror transfer method in the related art;

[0022] Figure 2 is a hardware structure block diagram of a mobile terminal of a mirror file synchronization method according to an embodiment of the application;

[0023] Figure 3 is a flowchart of a mirror file synchronization method according to an embodiment of the application;

[0024] Figure 4 is a mirror synchronization flowchart between a container mirror warehouse and a mirror database according to an embodiment of the application;

[0025] Figure 5 is a mirror synchronization flowchart between a container mirror warehouse and a mirror database according to an embodiment of the application;

[0026] Figure 6 is a mirror synchronization flowchart between a container mirror warehouse and a mirror database according to an embodiment of the application;

[0027] Figure 7 is a structure block diagram of a mirror file synchronization device according to an embodiment of the application;

[0028] Figure 8 is a structure diagram of an electronic device according to an embodiment of the application. DETAILED DESCRIPTION

[0029] In the following, the embodiments of the application will be described in detail with reference to the accompanying drawings and in conjunction with the embodiments.

[0030] It should be noted that the terms "first", "second" and the like in the specification and claims of the application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence.

[0031] The method embodiments provided in the embodiments of the application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking the case of running on a mobile terminal, Figure 2 is a hardware structure block diagram of a mobile terminal of a mirror file synchronization method according to an embodiment of the application. As Figure 2 shown, the mobile terminal can include one or more Figure 2The mobile terminal can further include a transmission device 206 for communication function and an input / output device 208. Those skilled in the art can understand that, Figure 2 The structure shown is only schematic, and does not limit the structure of the mobile terminal. For example, the mobile terminal can include more or less components than those shown, or have a different configuration of components. The same can also be true of the other figures described herein. Figure 2 The mobile terminal can include more or less components than those shown, or have a different configuration of components. Figure 2 The mobile terminal can include more or less components than those shown, or have a different configuration of components.

[0032] The memory 204 can be used to store computer programs, such as software programs of application software and modules, such as a computer program corresponding to the synchronization method of the mirror image file in the embodiments of the present application. The processor 202 can execute various functional applications and data processing by running the computer programs stored in the memory 204, i.e., implement the method described above. The memory 204 can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory 204 can further include a memory remotely arranged with respect to the processor 202, which can be connected to the mobile terminal through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0033] The transmission device 206 is used to receive or send data via a network. Specific examples of the network can include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 206 includes a network adapter (NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 206 can be a radio frequency (RF) module, which is used to communicate with the Internet in a wireless manner.

[0034] In the embodiments of the present application, a synchronization method of a mirror image file running on the mobile terminal or the network architecture is provided, Figure 3 is a flowchart of the synchronization method of the mirror image file according to the embodiments of the present application, as shown in Figure 3 The flowchart includes the following steps:

[0035] In step S302, in response to a synchronization instruction, the mirror image file to be synchronized is synchronized from the first container mirror image warehouse to the second container mirror image warehouse.

[0036] In step S302, the first container image warehouse is used to store the image file generated by the training platform for the preset training scene, and the second container image warehouse is used to store the image file required by the inference platform to provide services for the target object. The first container image warehouse and the second container image warehouse belong to the same type of image warehouse, but the first container image warehouse is different from the second container image warehouse. The image file represents the logic of the service provided by the inference platform, for example, the image file can be the logic of the inference platform providing shopping services to the shopping platform. The target object can be an individual or an enterprise, which is not limited in the present application.

[0037] Optionally, in step S302, the container image warehouse can be, but is not limited to, Harbor. Harbor is an open source container image warehouse. Harbor is based on Docker Registry and has corresponding enterprise-level extensions, so that Harbor has more extensive applications. These new enterprise-level features include, but are not limited to, a management user interface, role-based access control, AD (Active Directory) / LDAP (Lightweight Directory Access Protocol) integration, and audit logs.

[0038] The training platform described above is a resource platform for artificial intelligence development for artificial intelligence enterprise training scenarios. It can realize containerized deployment, visual development, centralized management and other functions, and can provide users with extremely high-performance AI (Artificial Intelligence) computing resources, realize efficient computing power support, accurate resource management and scheduling, agile data integration and acceleration, and process AI scene and business integration, effectively connecting development environment, computing resources and data resources, and improving development efficiency.

[0039] The inference platform described above can be, but is not limited to, an artificial intelligence inference server platform. It mainly faces enterprise AI application deployment and online service management scenarios. Through unified application interface, computing power elastic scaling, A / B testing, rolling release, multi-model weighted evaluation and other full-stack AI capabilities, it provides a reliable, easy-to-use and flexible inference service deployment and computing resource management platform for enterprises, helps users quickly put their AI business online, improves the utilization efficiency of AI computing resources, and realizes the rapid landing of AI industry.

[0040] In one optional embodiment, after receiving a synchronization command for the synchronized image files sent by the user, the inference platform can determine the source image repository (i.e., the first container image repository) and the target image repository (i.e., the second container image repository) based on the synchronization command. Then, the inference platform can use the cross-data warehouse image remote replication function supported by Harbor to synchronize the image files between the source image repository and the target image repository.

[0041] It's important to note that during the image file generation process on the training platform, developers frequently need to modify the image file. Once training is complete, a stable image file will be generated. At this point, developers can deploy the trained environment to the actual production environment, i.e., the inference platform. Image file synchronization at this stage requires the following steps: saving the image file from the server to the local machine, and then uploading the image file from the local machine to the inference platform for use. Typically, the image file generated by the training platform can reach 10GB or more, making the synchronization process lengthy. Furthermore, if any errors occur during the copying to the local machine or uploading to the inference platform, the image file needs to be resynchronized, further increasing the synchronization time.

[0042] In this application, because Harbor supports remote image replication across data warehouses, there is no need to save the image files locally or copy the local image to the inference platform during the synchronization of image files across image repositories. Even with large image files, synchronization between different container image repositories can be achieved quickly, thus improving the synchronization efficiency of image files. Even if any errors occur during the synchronization process, resynchronization will not significantly reduce synchronization efficiency.

[0043] Step S304: Traverse the image files in the second container image repository and the image files in the target database to obtain the traversal results. The target database is used to store image files that the inference platform can recognize.

[0044] In step S304, after achieving cross-repository synchronization of the container image repository, the inference platform needs to synchronize the image files in the container image repository of the inference platform to the image database of the inference platform (i.e., the target database mentioned above) before the inference platform can use these image files to provide services to users or enterprises.

[0045] In the process of synchronizing the image files in the second container image warehouse to the target database, the inference platform does not need to copy all the image files in the second container image warehouse to the target database, but traverses the second container image warehouse and the target database, determines the difference between the image files in the second container image warehouse and the image files in the target database according to the traversal result, and then synchronizes the image files in the second container image warehouse according to the difference.

[0046] In step S306, the target image file is determined according to the traversal result, and the target image file is obtained.

[0047] In step S306, the target image file can be an image file that the second container image warehouse has but the target database does not have, or an image file that the second container image warehouse does not have but the target database has.

[0048] It is easy to note that the number of image files that exist between the second container image warehouse and the target database is usually less than the number of image files that the second container image warehouse has, so in the process of synchronizing the container image warehouse of the inference platform and the image database of the inference platform, the synchronization of the image files that exist is can greatly reduce the synchronization efficiency of the image files.

[0049] In step S308, the image files in the second container image warehouse are synchronized to the target database based on the target image file, so that the inference platform uses the image files in the target database to provide services to the target object.

[0050] In step S308, the inference platform can determine the synchronization mode of the image files according to the file type corresponding to the target image file in the process of synchronizing the image files, wherein the file type corresponding to the target image file can include but is not limited to a first file type and a second file type, the first file type is an image file that the second container image warehouse has but the target database does not have, and the second file type is an image file that the second container image warehouse does not have but the target database has. That is, the synchronization mode corresponding to different file types is different, for example, for the first file type, the corresponding synchronization mode is to copy the image file that the second container image warehouse has but the target database does not have to the target database; for the second file type, the corresponding synchronization mode is to delete the image file that the second container image warehouse does not have but the target database has from the target database.

[0051] In addition, in the process of synchronizing the image files in the second container image warehouse to the target database, the target database is synchronized based on the differences between the image files in the second container image warehouse and the target database, and all the image files in the second container image warehouse are not copied, so that in the process of synchronizing the second container image warehouse and the target database, only the image files with differences need to be synchronized, and the synchronization efficiency of the image files is further improved. Moreover, even if any error occurs in the synchronization process, the re-synchronization will not significantly reduce the synchronization efficiency.

[0052] As can be known from the contents defined in the steps S302 to S308, since the container image warehouse has the "copy" feature, in the process of synchronizing the image files in different container image warehouses, the synchronization between the different container image warehouses can be realized directly based on the "copy" feature of the container image warehouse, without the need to save the image files to the local or upload the image files to the inference platform. When the image files are large, the synchronization between the different container image warehouses can also be quickly realized, thereby improving the synchronization efficiency of the image files. In addition, in the process of synchronizing the image files in the container image warehouse of the inference platform to the image database of the inference platform, the image database is synchronized based on the differences between the image files in the container image warehouse and the image database, and all the image files in the container image warehouse are not copied. Therefore, in the process of synchronizing the container image warehouse of the inference platform and the image database of the inference platform, only the image files with differences need to be synchronized, and the synchronization efficiency of the image files is further improved. Moreover, even if any error occurs in the synchronization process, the re-synchronization will not significantly reduce the synchronization efficiency.

[0053] It can be seen that the scheme provided in the present application can solve the problem of low synchronization efficiency of image files in the related art, and achieve the effect of improving the synchronization efficiency of image files.

[0054] It should be noted that the execution subject of the above steps can be the inference platform, but is not limited thereto.

[0055] In addition, it should be further noted that the steps S302 to S308 are described in detail as follows.

[0056] In an optional embodiment, the synchronization of the image files includes two stages, i.e., an image transfer stage and a database synchronization stage. The image transfer stage mainly realizes the synchronization of the image files between the container image warehouse of the inference platform and the container image warehouse of other platforms. The other platforms can be other inference platforms or training platforms. The database synchronization stage mainly realizes the synchronization of the image files between the container image warehouse (i.e., the second container image warehouse) of the inference platform and the image database (i.e., the target database) of the inference platform.

[0057] In the mirror transfer stage, the inference platform synchronizes the mirror file to be synchronized from the first container mirror warehouse to the second container mirror warehouse in response to the synchronization instruction.

[0058] Specifically, the inference platform parses the synchronization instruction to determine the project identifier of the source project, the project identifier of the target project, and the user information of the target user, and determines the first container mirror warehouse according to the project identifier of the source project and the second container mirror warehouse according to the project identifier of the target project. Then, according to the user information of the target user, the connection between the first container mirror warehouse and the second container mirror warehouse is established, and the synchronization permission of the target user is determined. Finally, when the synchronization permission of the target user reaches the target permission, the inference platform synchronizes the mirror file to be synchronized from the first container mirror warehouse to the second container mirror warehouse.

[0059] In the above process, the target user is the user who issues the synchronization instruction, and the synchronization permission represents whether the target user can access the mirror file in the first container mirror warehouse. Wherein, the synchronization permission represents whether the copy strategy of Harbor is activated, that is, in this embodiment, only the user with certain permission can activate the copy strategy of Harbor after issuing the synchronization instruction, so as to realize the mirror synchronization across the container mirror warehouses.

[0060] Optionally, the target user can determine the project identifier of the source project, the project identifier of the target project, and the username and password required for the connection between the container mirror warehouse corresponding to the source project and the container mirror warehouse corresponding to the target project, as well as the address of the container warehouse mirror and other information (i.e. the above-mentioned user information) through the terminal device. After the target user sets the above information, the terminal device can generate a synchronization instruction based on the above information, and send the above synchronization instruction to the inference platform, and then the inference platform parses the synchronization instruction to obtain the project identifier of the source project, the project identifier of the target project, and the user information of the target user. The inference platform can determine the address of the container mirror warehouse of the source project and the address of the container mirror warehouse of the target project according to the project identifier of the source project and the project identifier of the target project, and establish the connection between the two container mirror warehouses based on the addresses of the two container mirror warehouses.

[0061] After the connection between the two container image repositories, the inference platform determines the permissions according to the user information of the target user, for example, the inference platform can determine whether the target user can access the image file in the first container image repository according to the username of the target user. When it is determined that the user can access the image file in the first container image repository, the inference platform determines that the replication strategy of Harbor is activated, at this time, the inference platform copies all image files under the source project of Harbor to the container image repository corresponding to the target project of Harbor, thereby realizing the cross-repository synchronization of the container image file.

[0062] It should be noted that, since Harbor supports the cross-data-repository image remote replication function, in the process of synchronizing the image file across the image repositories, there is no need to save the image file to the local, nor to copy the local image to the inference platform, and when the image file is large, the synchronization between different container image repositories can also be quickly realized, thereby improving the synchronization efficiency of the image file. Even if any error occurs during the synchronization process, re-synchronization will not significantly reduce the synchronization efficiency.

[0063] Further, after synchronizing the image file to be synchronized from the first container image repository to the second container image repository, when the image file in the first container image repository is updated, the inference platform synchronizes the updated image file from the first container image repository to the second container image repository.

[0064] It should be noted that, since Harbor has the function of cross-data-repository image remote replication, when the image file under the source project of Harbor is added or deleted, as long as the replication strategy is still in the activated state, the changes of the image file in the first container image repository will be automatically synchronized to the second container image repository, without the need for the user to manually operate, realizing the cross-image-repository update of the image file and improving the synchronization efficiency of the image file.

[0065] In an optional embodiment, in the database synchronization stage, the inference platform performs steps S304 to S308 to realize the synchronization of the image between the container image repository of the inference platform and the image database. In this stage, the inference platform can determine the synchronization mode of the image file according to the file type corresponding to the target image file.

[0066] When the file type is the first file type (i.e. the image file possessed by the second container image repository but not possessed by the target database), the inference platform traverses the image files in the second container image repository and the image files in the target database to obtain a traversal result, and obtains the target file identifier of the first image file in the second container image repository and the target file size of the first image file according to the traversal result. When there is no image file in the target database that has the same file identifier as the target file identifier of the first image file and has the same file size as the target file size of the first image file, the first image file is determined to be the first target image file.

[0067] After determining that the target image file is the first target image file, the inference platform obtains service information of a service provided by the inference platform, a file identifier of the first target image file, image synchronization information of the first target image file, and generates target data based on the service information, the file identifier of the first target image file, and the image synchronization information, and then synchronizes the target data and the first target image file to the target database.

[0068] Optionally, Figure 4 The image synchronization process between the container image repository and the image database when the file type is the first file type (i.e. the image file possessed by the second container image repository but not possessed by the target database) is shown. As shown in Figure 4 The inference platform first traverses the image files in the second container image repository and the image files in the target database, and detects whether there is an image file in the target database that has the same file identifier as the target file identifier of the first image file and has the same file size as the target file size of the first image file. If there is, the inference platform continues to compare the next image file in the second container image repository with the image files in the target database until the image files in the second container image repository are the same as the image files in the target database, or there is no image file in the target database that has the same file identifier as the target file identifier of the first image file or has the same file size as the target file size of the first image file.

[0069] In addition, as Figure 4As shown, when there exists a mirror image file in the target database with the same file identification as the target file identification of the first mirror image file and the same file size as the target file size of the first mirror image file, the inference platform combines the file identification (for example, the name of the mirror image file) of the target mirror image file, the service information of the service provided by the inference platform, and the mirror image synchronization information of the target mirror image file according to the data format recognizable by the inference platform to obtain target data, and inserts the target data into the target database of the inference platform, while synchronizing the first mirror image file to the target database of the inference platform, so that not only the first mirror image file but also the related information of the first mirror image file is stored in the target database of the inference platform.

[0070] It should be noted that the service information of the service provided by the inference platform can include but is not limited to object information of an object receiving the service, for example, object name, user group to which the object belongs, etc., wherein the object information represents the permissions of the object, and the permissions represent the services provided by the inference platform that the object can enjoy, for example, the inference platform provides services A and B to object 1, that is, object 1 can access the mirror image files corresponding to services A and B; the inference platform provides services A and C to object 2, that is, object 2 can access the mirror image files corresponding to services A and C.

[0071] In addition, the mirror image synchronization information can include but is not limited to the time of synchronizing the mirror image file between the second container mirror image repository and the target database, and the size of the target mirror image file, etc.

[0072] When the file type is the second file type (that is, the second container mirror image repository does not have but the target database has the mirror image file), the inference platform traverses the mirror image files in the second container mirror image repository and the mirror image files in the target database to obtain a traversal result, and obtains the target file identification of the second mirror image file in the target database and the target file size of the second mirror image file according to the traversal result. When there is no mirror image file in the second container mirror image repository with the same file identification as the target file identification of the second mirror image file and the same file size as the target file size of the second mirror image file, the inference platform determines that the second mirror image file is the second target mirror image file.

[0073] After determining the target mirror image file, the inference platform deletes the second target mirror image file from the target database.

[0074] Optionally, Figure 5 As shown, when the file type is the second file type (that is, the second container mirror image repository does not have but the target database has the mirror image file), the mirror image synchronization process between the container mirror image repository and the mirror image database is shown. As shown in Figure 5As shown, the reasoning platform first traverses the image files in the second container image repository and traverses the image files in the target database, and detects whether there is an image file in the second container image repository that has the same file identifier as the target file identifier of the second image file and has the same file size as the target file size of the second image file. If there is, the reasoning platform continues to compare the next image file in the target database with the image file in the second container image repository until the image files in the target database and the image files in the second container image repository are completely the same, or there is no image file in the second container image repository that has the same file identifier as the target file identifier of the second image file and has the same file size as the target file size of the second image file.

[0075] In addition, as Figure 5 As shown, when there is no image file in the second container image repository that has the same file identifier as the target file identifier of the second image file and has the same file size as the target file size of the second image file, the reasoning platform deletes the second target image file and the related data corresponding to the second target image file from the target database.

[0076] It should be noted that the above method can ensure that the user's original data in the reasoning platform is not deleted, and the images of other projects can be synchronized, thereby realizing the synchronization of the container image repository and the image database of the reasoning platform.

[0077] In addition, it should be noted that in actual application, the reasoning platform needs to perform the above two processing methods of the first file type and the second file type in the process of the image files between the container image repository and the image database. The above two processing methods of the first file type and the second file type can be performed simultaneously, or the execution order can be set according to the requirements. In this embodiment, the order of the two processing methods is not specifically limited, for example, Figure 6 The synchronization process of the image files is schematically shown, in which the processing method of the first file type is performed first, and then the processing method of the second file type is performed.

[0078] In an alternative embodiment, the user can slightly modify the scheme provided in this embodiment, and the synchronization of the images between the reasoning platform and other training platforms can be realized.

[0079] Specifically, when detecting that the image file in the second container image warehouse has an update, the updated image file is acquired, and the updated image file is synchronized to the first container image warehouse. For example, in actual application, the inference platform updates (for example, modifies, deletes, or increases) the image file in the second container image warehouse according to actual needs, at this time, the inference platform can synchronize the updated image file to the container image warehouse (that is, the first container image warehouse) of the training platform, thereby realizing synchronization of the images of the first container image warehouse and the second container image warehouse.

[0080] It should be noted that the image synchronization process between the inference platform and other training platforms is similar to the process of synchronizing the images from the training platform to the inference platform, which will not be described here.

[0081] In another optional embodiment, the scheme provided in the embodiment can also be applied to image transfer between inference platforms. Specifically, after synchronizing the image file in the second container image warehouse to the target database based on the target image file, the inference platform synchronizes the image file corresponding to the first inference platform to the container image warehouse corresponding to the second inference platform in response to a platform synchronization instruction, wherein the container image warehouse of the first inference platform is the same as the container image warehouse of the second inference platform.

[0082] Optionally, after receiving the synchronization instruction, the source inference platform (that is, the first inference platform) analyzes the synchronization instruction, that is, determines the related information of the target inference platform (that is, the second inference platform), for example, the address, name, image access permission of the target inference platform, and then the source inference platform determines the image file to be synchronized according to the image access permission of the target inference platform, and synchronizes the image file to be synchronized to the container image warehouse of the target inference platform according to the address, name, and the like of the target inference platform.

[0083] From the above content, it can be known that the scheme provided in the embodiment can realize synchronization of the image file from the training environment or the user test environment to the inference platform, and also realize synchronization of the image file between the inference platforms, which is simple to operate, greatly improves the synchronization speed of the image file, and has strong portability after slight modification of the scheme.

[0084] Those skilled in the art can clearly understand that the method according to the above-mentioned embodiments can be realized by means of software and necessary general hardware platforms, of course, it can also be realized by hardware, but in many cases, the former is a better implementation. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server, or network device, etc.) execute the method described in each embodiment of the present application.

[0085] In the embodiment, a synchronization device of the mirror image file is also provided, which is used to realize the above-mentioned embodiments and preferred embodiments, and will not be described again. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably realized in software, hardware, or a combination of software and hardware is also possible and is contemplated.

[0086] Figure 7 is a structural block diagram of the synchronization device of the mirror image file according to the embodiment of the present application, as shown in Figure 7 , the device comprises a first synchronization module 701, a traversal module 703, a difference determination module 705 and a second synchronization module 707.

[0087] The first synchronization module 701 is configured to synchronize the mirror image file to be synchronized from the first container image warehouse to the second container image warehouse in response to a synchronization instruction, the first container image warehouse is configured to store the mirror image file generated by the training platform for the preset training scene, the second container image warehouse is configured to store the mirror image file required by the inference platform to provide services for the target object, and the first container image warehouse and the second container image warehouse belong to the same type of image warehouse; the traversal module 703 is configured to traverse the mirror image file in the second container image warehouse and traverse the mirror image file in the target database to obtain a traversal result, wherein the target database is configured to store the mirror image file that can be recognized by the inference platform; the difference determination module 705 is configured to determine the mirror image file with differences between the second container image warehouse and the target database according to the traversal result to obtain a target mirror image file; and the second synchronization module 707 is configured to synchronize the mirror image file in the second container image warehouse to the target database based on the target mirror image file, so that the inference platform uses the mirror image file in the target database to provide services for the target object.

[0088] Optionally, the first synchronization module comprises a first determination module, a second determination module, a third determination module and a third synchronization module. The first determination module is configured to parse the synchronization instruction, and determine the project identifier of the source project, the project identifier of the target project and the user information of the target user, wherein the target user is the user who sends the synchronization instruction. The second determination module is configured to determine the first container image warehouse according to the project identifier of the source project, and determine the second container image warehouse according to the project identifier of the target project. The third determination module is configured to establish the connection between the first container image warehouse and the second container image warehouse according to the user information of the target user, and determine the synchronization permission of the target user, wherein the synchronization permission represents whether the target user can access the image file in the first container image warehouse. The third synchronization module is configured to synchronize the image file to be synchronized from the first container image warehouse to the second container image warehouse when the synchronization permission of the target user reaches the target permission.

[0089] Optionally, the image file synchronization device further comprises a fourth synchronization module, configured to synchronize the updated image file from the first container image warehouse to the second container image warehouse when the image file in the first container image warehouse is updated after the image file to be synchronized is synchronized from the first container image warehouse to the second container image warehouse.

[0090] Optionally, the difference determination module comprises a first acquisition module, a fourth determination module, a third acquisition module and a fifth determination module. The first acquisition module is configured to acquire the target file identifier and the target file size of the first image file in the second container image warehouse according to the traversal result. The fourth determination module is configured to determine that the first image file is the first target image file when there is no image file with the same file identifier and the same file size as the target file identifier and the target file size in the target database. The third acquisition module is configured to acquire the target file identifier and the target file size of the second image file in the target database according to the traversal result. The fifth determination module is configured to determine that the second image file is the second target image file when there is no image file with the same file identifier and the same file size as the target file identifier and the target file size in the second container image warehouse.

[0091] Optionally, the second synchronization module comprises a second acquisition module, a generation module and a fifth synchronization module. The second acquisition module is configured to acquire the service information of the service provided by the inference platform, the file identifier of the target image file and the image synchronization information of the target image file when the target image file is the first target image file. The generation module is configured to generate target data based on the service information, the file identifier of the target image file and the image synchronization information. The fifth synchronization module is configured to synchronize the target data and the target image file into the target database.

[0092] Optionally, the second synchronization module comprises a file deletion module configured to delete the target image file from the target database when the target image file is a second target image file.

[0093] Optionally, the image file synchronization apparatus further comprises a fourth acquisition module and a sixth synchronization module. The fourth acquisition module is configured to acquire an updated image file when it is detected that the image file in the second container image repository has an update. The sixth synchronization module is configured to synchronize the updated image file to the first container image repository.

[0094] Optionally, the image file synchronization apparatus further comprises a seventh synchronization module configured to synchronize the image file corresponding to the first inference platform to the container image repository corresponding to the second inference platform in response to a platform synchronization instruction after the image file in the second container image repository is synchronized to the target database based on the target image file, wherein the container image repository of the first inference platform is the same as the container image repository of the second inference platform.

[0095] It should be noted that each of the above modules can be implemented by software or hardware, and for the latter, the following implementation manners can be used, but are not limited thereto: all the above modules are located in the same processor; or the above modules are located in different processors in any combination.

[0096] Embodiments of the present application also provide a computer readable storage medium having a computer program stored therein, wherein the computer program is configured to execute the steps in any of the above method embodiments when running.

[0097] In an example embodiment, the above computer readable storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0098] Embodiments of the present application also provide an electronic device, wherein Figure 8 is a schematic diagram of an optional electronic device according to an embodiment of the present application, as Figure 8 shown, comprising a memory and a processor, the memory having a computer program stored therein, and the processor being configured to run the computer program to execute the steps in any of the above method embodiments.

[0099] In an example embodiment, the above electronic device can further comprise a transmission device and an input / output device Figure 8The input / output device (not shown) is connected to the processor and the transmission device is connected to the processor.

[0100] The specific examples in the embodiments can refer to the examples described in the above embodiments and exemplary embodiments, which will not be repeated here.

[0101] Obviously, those skilled in the art should understand that the modules or steps of the present application described above can be realized by general computing devices, which can be concentrated on a single computing device or distributed on a network composed of multiple computing devices, which can be realized by program codes executable by the computing devices, so that they can be stored in storage devices and executed by the computing devices, and in some cases, the steps shown or described can be executed in different order, or they can be made into individual integrated circuit modules, or multiple modules or steps can be made into a single integrated circuit module. Thus, the present application is not limited to any specific combination of hardware and software.

[0102] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Those skilled in the art can make various modifications and changes to the present application. Any modification, equivalent replacement, improvement, etc. within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for synchronizing image files, characterized in that, include: In response to the synchronization command, the image file to be synchronized is synchronized from the first container image repository to the second container image repository. The first container image repository is used to store the image files generated by the training platform for the preset training scenario, and the second container image repository is used to store the image files required by the inference platform to provide services to the target object. The first container image repository and the second container image repository are image repositories of the same type. The image files in the second container image repository are traversed, and the image files in the target database are traversed to obtain the traversal results. The target database is used to store image files that the inference platform can recognize. Based on the traversal results, image files that differ between the second container image repository and the target database are determined, and the target image file is obtained; Based on the target image file, the image files in the second container image repository are synchronized to the target database, so that the inference platform can use the image files in the target database to provide services to the target object.

2. The method according to claim 1, characterized in that, In response to the synchronization command, the image files to be synchronized are synchronized from the first container image repository to the second container image repository, including: The synchronization command is parsed to determine the project identifier of the source project, the project identifier of the target project, and the user information of the target user, wherein the target user is the user who issued the synchronization command; The first container image repository is determined based on the project identifier of the source project, and the second container image repository is determined based on the project identifier of the target project. A connection is established between the first container image repository and the second container image repository based on the user information of the target user, and the synchronization permissions of the target user are determined, wherein the synchronization permissions indicate whether the target user can access the image files in the first container image repository; When the target user's synchronization permissions reach the target permissions, the image file to be synchronized is synchronized from the first container image repository to the second container image repository.

3. The method according to claim 2, characterized in that, After synchronizing the image file to be synchronized from the first container image repository to the second container image repository, the method further includes: When an image file in the first container image repository is updated, the updated image file is synchronized from the first container image repository to the second container image repository.

4. The method according to claim 1, characterized in that, Based on the traversal results, image files that differ between the second container image repository and the target database are determined, resulting in the target image file, including: Based on the traversal results, obtain the target file identifier and target file size of the first image file in the second container image repository; if there is no image file in the target database with the same file identifier as the target file identifier of the first image file and the same file size as the target file size of the first image file, determine the first image file as the first target image file; Based on the traversal results, obtain the target file identifier and target file size of the second image file in the target database; if there is no image file in the second container image repository with the same file identifier as the target file identifier of the second image file and the same file size as the target file size of the second image file, then determine the second image file as the second target image file.

5. The method according to claim 4, characterized in that, Synchronizing the image files in the second container image repository to the target database based on the target image file includes: When the target image file is the first target image file, obtain the service information provided by the inference platform, the file identifier of the first target image file, and the image synchronization information of the first target image file; Target data is generated based on the service information, the file identifier of the first target image file, and the image synchronization information. The target data and the first target image file are synchronized to the target database.

6. The method according to claim 4, characterized in that, Synchronizing the image files in the second container image repository to the target database based on the target image file includes: When the target image file is the second target image file, the second target image file is deleted from the target database.

7. The method according to claim 1, characterized in that, The method further includes: When an update is detected in the image file of the second container image repository, the updated image file is retrieved; The updated image file is synchronized to the first container image repository.

8. The method according to claim 1, characterized in that, After synchronizing the image files in the second container image repository to the target database based on the target image file, the method further includes: In response to the platform synchronization command, the image file corresponding to the first inference platform is synchronized to the container image repository corresponding to the second inference platform, wherein the container image repository of the first inference platform is the same as the container image repository of the second inference platform.

9. A device for synchronizing image files, characterized in that, include: The first synchronization module is used to respond to the synchronization command and synchronize the image file to be synchronized from the first container image repository to the second container image repository. The first container image repository is used to store the image file generated by the training platform for the preset training scenario, and the second container image repository is used to store the image file required by the inference platform to provide services to the target object. The first container image repository and the second container image repository are image repositories of the same type. The traversal module is used to traverse the image files in the second container image repository and the image files in the target database to obtain the traversal results. The target database is used to store image files that the inference platform can recognize. The difference determination module is used to determine, based on the traversal results, the image files that differ between the second container image repository and the target database, and obtain the target image file; The second synchronization module is used to synchronize the image files in the second container image repository to the target database based on the target image file, so that the inference platform can use the image files in the target database to provide services to the target object.

10. A computer-readable storage medium, characterized in that, A computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the method for synchronizing the image file according to any one of claims 1 to 8.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method for synchronizing the image file as described in any one of claims 1 to 8.

Images