A high-concurrency image transmission method based on Openstack

Abstract

The invention discloses a high-concurrency image transmission method based on Openstack, including: creating an Openstack cluster with OVN network technology; storing the image to be uploaded in the image library, and managing the image in the image library; establishing an image library management page for selecting the image to be uploaded and all target physical machines; determining the priority of the image to be uploaded according to the AHP hierarchical analysis method, selecting an image to be uploaded from the image library according to the priority order, dividing the image to be uploaded into image fragments, creating a corresponding number of virtual machines in the Openstack cluster according to the number of copies of the image fragments, and realizing the storage and sending of the image fragments; selecting a physical machine for the sent image fragment according to the high-concurrency technology, and receiving the image fragments; recombining the image fragments on each physical machine into a complete image. The image transmission method of the invention improves the image transmission speed.

Description

Technical Field

[0001] The present invention relates to the technical field of network mirroring high-speed transmission, and in particular to a high-concurrency mirroring transmission method based on Openstack. Background Art

[0002] OpenStack is a cloud platform management project with a huge application in the field of cloud computing. It is one of the current mainstream cloud computing technologies with market prospects and is suitable for many application scenarios, such as business and marketing. With a large enough market share, its smooth operation is inseparable from the credit of operation and maintenance.

[0003] Image transmission is very important in Openstack. It can be said that without this technology, it is difficult to create various virtual machines on the Openstack cloud management platform. The same physical machine can create 10-100 virtual machines, and these virtual machines have various functions. The fundamental reason why virtual machines have such various functions is that when pulling up virtual machines on Openstack, they rely on various different images. If the virtual machine is compared to a computer, the image is equivalent to a collection of files that store all the operating programs of the computer. The biggest feature of the image is that if a specific code is encapsulated in the image, the created virtual machine can directly run the code. At the same time, the above services can also be customized and provided to any organization for use.

[0004] The transmission of images has greatly expanded the functions of the physical machine, because the more types of images there are, the more types of virtual machines the physical machine can provide. However, at present, when the scale of Openstack is large enough, the types of images are also very rich. Usually, images are stored in the image library and transmitted to the physical machine and the images in the image library at any time through network technology. However, traditional transmission methods are limited by the quality of bandwidth. It is difficult to quickly transmit images that are too large, which will cause a lot of losses. Summary of the invention

[0005] In view of the problems existing in the prior art, the present invention provides a high-concurrency image transmission method based on Openstack, which provides connection and resource allocation with high concurrency, and the connection mode between virtual machines and physical machines provides cache and file reorganization, which can be used to transmit images of any size.

[0006] To achieve the above object, the present invention adopts the following technical solution: a high-concurrency mirror transmission method based on Openstack, which specifically includes the following steps:

[0007] Step 1. Create an Openstack cluster with OVN network technology;

[0008] Step 2: Store the image to be uploaded in the image library and manage the images in the image library;

[0009] Step 3: Establish an image library management page for selecting the image to be uploaded and all target physical machines. The network connection method of the image library management page adopts the flow table of the OVN network technology for connection;

[0010] Step 4: Determine the priority of the image to be uploaded according to the AHP hierarchical analysis method, select an image to be uploaded from the image library according to the priority order, divide the image to be uploaded into image fragments, and create a corresponding number of virtual machines in the Openstack cluster according to the number of image fragments to realize the storage and transmission of the image fragments;

[0011] Step 5: Select a physical machine for the sent image fragments according to the high concurrency technology and receive the image fragments;

[0012] Step 6: Execute steps 4 and 5 for all images to be uploaded.

[0013] Step 7. Reassemble the image fragments on each physical machine and form a complete qemu image according to the qemu-img convert-fqcow2kwxcc.qcow2-O vmdk kwxcc.vmdk technology.

[0014] Furthermore, the Openstack cluster is 3-control 3-computing, and each virtual machine in the Openstack cluster is configured in a haproxy high-availability manner.

[0015] Furthermore, the image library management page is implemented through Docker technology. Docker technology provides a Horizon container for the image library. A page management system is deployed inside the container to select the image to be uploaded and all target physical machines; and an image management system is developed on the image library management page to read all the images in the image library.

[0016] Furthermore, the specific process of determining the priority of the image to be uploaded according to the AHP hierarchical analysis method in step S4 is as follows:

[0017] (A) Use the docker images command on the image library management page to query the size of the image to be uploaded in the image library, find the maximum and minimum image sizes, divide the image size range into 10 sub-ranges, arrange the sub-ranges from small to large, and divide each image to be uploaded into the corresponding sub-range;

[0018] (B) Set the weight level from 1 to 10, where 1 represents the highest level and 10 represents the lowest level. The value multiplied by the weight level and the image size in the corresponding sub-interval is used as the priority of the image to be uploaded. The smaller the value, the higher the priority.

[0019] Furthermore, in step 4, the slowest cache time is set in the created virtual machine to increase the delay time of 10-30s based on the slowest upload time of the virtual machine.

[0020] Furthermore, step 5 includes the following sub-steps:

[0021] (5.1) Physical machines are divided into 1-10 levels according to business requirements, 1 represents the highest business level and 10 represents the lowest business level. At the same time, network speed is divided into 1-3 levels, 1 represents stable network and good speed, 2 represents quantitative network, and 3 represents slow speed.

[0022] (5.2) The value obtained by multiplying the network speed level and the service level of each physical machine is used as the priority of the corresponding physical machine. The smaller the value, the higher the priority.

[0023] (5.3) The sent image fragments are received by the high-concurrency virtual network built by OVN according to the priority of the physical machine.

[0024] Compared with the prior art, the present invention has the following beneficial effects: the high-concurrency image transmission method based on Openstack of the present invention adopts high concurrency for image transmission, which needs to rely on stable and highly secure high-concurrency technology, and realizes the ability to assemble image fragments into a complete image through the cross-node forwarding cache technology of the host image file. The high-concurrency image transmission method based on Openstack of the present invention relies on high-concurrency technology, determines the priority of the image to be uploaded through the AHP hierarchical analysis method, and then divides the image to be uploaded into image fragments, and different physical machines are responsible for the transmission of one image fragment. This process is executed concurrently, and under the same network bandwidth, the speed of image transmission can be greatly improved; at the same time, the physical machine selection is performed for the image fragment to be sent, the image fragment is received, and finally the image fragments on each physical machine are reassembled into a complete image file, which improves the transmission efficiency compared with single image transmission and transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 The present invention is a flowchart of a high-concurrency image transmission method based on Openstack. DETAILED DESCRIPTION

[0026] The technical solution of the present invention is further explained below in conjunction with the accompanying drawings.

[0027] like Figure 1The flowchart of the high-concurrency image transmission method based on Openstack of the present invention is as follows:

[0028] Step 1, create an Openstack cluster with OVN network technology; in the present invention, the Openstack cluster is 3-control 3-computing, that is, it needs to pass through three identical Openstack cloud control systems. These three physical machines jointly form a complete image file upload system, which provides hardware and software support for the creation of virtual machines; each virtual machine in the Openstack cluster is configured in a haproxy high-availability manner, which can achieve high concurrency in the process and realize efficient and stable operation of virtual machine services. The role of haproxy is a protection mechanism to ensure that when the virtual machine is running, due to the sudden downtime of the physical machine, the virtual machine function of the downtime node is re-executed on other physical machine nodes to ensure the stable upload of the image fragment. The OVN network adopts the method of issuing flow tables to realize the network connection between physical machines. This connection method can provide a very stable network service for uploading image files.

[0029] Step 2: Store the image to be uploaded in the image library and manage the images in the image library;

[0030] Step 3, establish an image library management page for selecting the image to be uploaded and all target physical machines. The network connection method of the image library management page in the present invention adopts the flow table of the OVN network technology for connection; the image library management page is implemented by Docker technology, and Docker technology provides a Horizon container for the image library. A page management system is deployed inside the container for selecting the image to be uploaded and all target physical machines; and an image management system is developed on the image library management page to read all images in the image library.

[0031] Step 4: Determine the priority of the image to be uploaded according to the AHP hierarchical analysis method, select an image to be uploaded from the image library according to the priority order, and divide the image to be uploaded into image fragments. In the present invention, the image to be uploaded can be divided into different image fragments through the Match API, and according to the number of image fragments, a corresponding number of virtual machines are created in the Openstack cluster to realize the storage and sending of the image fragments. In the present invention, the image fragments can be sent through the Return API; Openstack will select different images as the core of virtual machine creation according to the needs of creating virtual machines, and different virtual machines can be created according to the images.

[0032] The specific process of determining the priority of the image to be uploaded according to the AHP hierarchical analysis method in the present invention is as follows:

[0033] (A) Use the docker images command on the image library management page to query the size of the image to be uploaded in the image library, find the maximum and minimum image sizes, divide the image size range into 10 sub-ranges, arrange the sub-ranges from small to large, and divide each image to be uploaded into the corresponding sub-range;

[0034] (B) Set the weight level from 1 to 10, where 1 represents the highest level and 10 represents the lowest level. The value multiplied by the weight level and the image size in the corresponding sub-interval is used as the priority of the image to be uploaded. The smaller the value, the higher the priority.

[0035] At the same time, considering that multiple virtual machines upload an image fragment at the same time, and the image fragments are different, once there is a network fluctuation, or the image is uploaded only according to the average upload time of the image, the image fragments of some virtual machines will not be uploaded completely. Even if the image merging technology can be used later, the merged image is not complete. Therefore, the slowest cache time is set in the created virtual machine to add a delay time of 10-30s on the basis of the slowest upload time of the virtual machine, so as to ensure that the slowest one among the multiple virtual machines can complete the image upload, so that the image obtained after the later reorganization is a complete image.

[0036] Step 5. After uploading the image to different physical machines, corresponding services need to be provided for the physical machines. In actual use, different requirements are required according to the frequency and importance of the business. For some high-priority services, if the image cannot be delivered to the designated location in time, it will cause the interruption of the business, causing economic losses in actual production. Therefore, it is very important to select a physical machine with high business priority to configure the required image in place in a timely and effective manner, so that the corresponding service can be pulled up according to the image. Therefore, according to the high concurrency technology, the physical machine selection is carried out for the sent image fragments, and the image fragments are received; the specific sub-steps include:

[0037] (5.1) Physical machines are divided into 1-10 levels according to business requirements, 1 represents the highest business level and 10 represents the lowest business level. At the same time, network speed is divided into 1-3 levels, 1 represents stable network and good speed, 2 represents quantitative network, and 3 represents slow speed.

[0038] (5.2) The value obtained by multiplying the network speed level and the service level of each physical machine is used as the priority of the corresponding physical machine. The smaller the value, the higher the priority.

[0039] (5.3) The sent image fragments are received by the high-concurrency virtual network built by OVN according to the priority of the physical machine.

[0040] The priority of physical machines is established based on the urgency of business needs and the speed of the network, and image fragments of different sizes are automatically allocated to physical machines with different bandwidths, striving to ensure that all image fragments are transmitted almost simultaneously; at the same time, high concurrency technology is a distributed file transfer technology that can divide image files into different fragments and then transmit them simultaneously. After the high-concurrency node confirms the network resources, the largest image fragment enjoys the resources of all physical machines according to the size of the image fragment. The image fragment that does not occupy a lot of memory is less likely to use a large physical machine, thereby ensuring the smooth execution of the receiving task.

[0041] Step 6: Execute steps 4 and 5 for all images to be uploaded.

[0042] Step 7. Reassemble the image fragments on each physical machine and form a complete qemu image according to the qemu-img convert-f qcow2kwxcc.qcow2-O vmdk kwxcc.vmdk technology.

[0043] The high-concurrency mirror transmission method based on Openstack of the present invention improves the mirror transmission speed, can promote the operator to provide users with virtual machine business functions to enable users to obtain virtual machine services more quickly; through efficient mirror transmission technology, it can provide high-quality virtual machine services. For IOT type services, different physical machines need to be provided by virtual machines. Services, connecting different hardware, and high-speed transmission of mirrors enable the same physical node to expand different virtual machine functions and connect more different IOT hardware. In the context of the strategic development of cloud-to-digital transformation, the present invention can effectively promote the development of cloud computing technology.

[0044] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A high-concurrency image transmission method based on Openstack, It is characterized in that The specific steps include: Step 1. Create an Openstack cluster with OVN network technology; Step 2: Store the image to be uploaded in the image library and manage the images in the image library; Step 3: Establish an image library management page for selecting the image to be uploaded and all target physical machines. The network connection method of the image library management page adopts the flow table of the OVN network technology for connection; Step 4: Determine the priority of the image to be uploaded according to the AHP hierarchical analysis method, select an image to be uploaded from the image library according to the priority order, divide the image to be uploaded into image fragments, and create a corresponding number of virtual machines in the Openstack cluster according to the number of image fragments to realize the storage and transmission of the image fragments; The specific process of determining the priority of the image to be uploaded according to the AHP hierarchical analysis method is as follows: (A) Use the docker images command on the image library management page to query the size of the image to be uploaded in the image library, find the maximum and minimum image sizes, divide the image size range into 10 sub-ranges, arrange the sub-ranges from small to large, and divide each image to be uploaded into the corresponding sub-range; (B) Set the weight level from 1 to 10, where 1 represents the highest level and 10 represents the lowest level. The value multiplied by the weight level and the image size in the corresponding sub-interval is used as the priority of the image to be uploaded. The smaller the value, the higher the priority. Step 5: Select a physical machine for the sent image fragments according to the high concurrency technology and receive the image fragments; Step 6: Execute steps 4 and 5 for all images to be uploaded. Step 7. Reassemble the image fragments on each physical machine and form a complete qemu image according to the qemu-img convert-fqcow2kwxcc.qcow2-O vmdk kwxcc.vmdk technology.

2. According to the high-concurrency mirror transmission method based on Openstack according to claim 1, It is characterized in that The Openstack cluster is a 3-control 3-computing cluster, and each virtual machine in the Openstack cluster is configured in a haproxy high availability manner.

3. According to the high-concurrency image transmission method based on Openstack in claim 1, It is characterized in that The image library management page is implemented through Docker technology. Docker technology provides a Horizon container for the image library. A page management system is deployed inside the container to select the image to be uploaded and all target physical machines. And develop an image management system on the image library management page to read all the images in the image library.

4. According to claim 1, a high-concurrency image transmission method based on Openstack, It is characterized in that In step 4, set the slowest cache time in the created virtual machine to add a delay of 10-30s based on the slowest upload time of the virtual machine.

5. According to the high-concurrency image transmission method based on Openstack in claim 1, It is characterized in that Step 5 includes the following sub-steps: (5.1) Physical machines are divided into 1-10 levels according to business requirements, 1 represents the highest business level and 10 represents the lowest business level. At the same time, network speed is divided into 1-3 levels, 1 represents stable network and good speed, 2 represents quantitative network, and 3 represents slow speed. (5.2) The value obtained by multiplying the network speed level and the service level of each physical machine is used as the priority of the corresponding physical machine. The smaller the value, the higher the priority. (5.3) The sent image fragments are received by the high-concurrency virtual network built by OVN according to the priority of the physical machine.

Images