VR-based panoramic display system and panoramic image splicing method

Abstract

The invention relates to a panoramic display system based on VR and a panoramic image stitching method. The system comprises a closed panoramic display screen and a video stitching server. The video stitching server is used for stitching a plurality of panoramic images to merge, generating a panoramic video, and outputting the panoramic video to the closed panoramic display screen for panoramic display; according to the invention, the closed panoramic display screen can display the content at 360 degrees without dead angles, and audiences can freely rotate the visual angle in the display screen to feel immersive experience. The first feature extraction algorithm and the second feature extraction algorithm are combined, the advantages of different algorithms can be fully utilized, and the accuracy and robustness of feature point extraction are improved. The feature points are screened according to the local gradient of the feature points, the difference of the descriptors and the positions of the feature points in the to-be-spliced image, the feature points which are unstable or do not meet requirements are removed, the feature points with higher representativeness and stability are reserved, and the matching accuracy and reliability are improved.

Description

Technical Field

[0001] The present invention relates to the field of image stitching, and in particular to a panoramic display system based on VR and a panoramic image stitching method. Background Art

[0002] VR panoramic video is a type of video content produced using virtual reality (VR) technology, which allows viewers to watch videos from a 360-degree panoramic perspective, as if they were in the scene presented in the video. This form of video provides viewers with a more immersive viewing experience and is widely used in tourism, education, entertainment and other fields.

[0003] However, panoramic videos require stitching images when stitching, but feature point extraction in the prior art is not accurate enough, and the VR panoramic video display effect is poor. Summary of the invention

[0004] In view of this, the present invention aims to propose a panoramic display system and a panoramic image stitching method based on VR, so as to solve the problem that feature point extraction is not accurate enough when stitching images in the prior art and the VR panoramic video display effect is poor in the prior art.

[0005] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0006] The first aspect of the present invention provides a VR-based panoramic display system, comprising:

[0007] Closed panoramic display screen and video splicing server;

[0008] The video stitching server is used to stitch multiple panoramic images to merge and generate a panoramic video, and output the panoramic video to a closed panoramic display screen for panoramic display.

[0009] Further, the closed panoramic display screen includes a plurality of LED display screens;

[0010] Each of the LED display screens forms a closed polyhedron to perform a panoramic display of the panoramic video output by the video splicing server.

[0011] A second aspect of the present invention provides a VR-based panoramic image stitching method applied to the first aspect, comprising:

[0012] Acquire multiple images to be stitched;

[0013] Based on the first feature extraction algorithm and the second feature extraction algorithm, feature points in each image to be stitched are respectively extracted to obtain a comprehensive feature point set;

[0014] Delete duplicate feature points in the comprehensive feature point set, and screen the feature points in the comprehensive feature point set according to the local gradient of the feature point, the difference of the descriptor and the position of the feature point in the image to be stitched;

[0015] Use transformation model to match feature point pairs;

[0016] The panoramic image is obtained by performing image fusion based on the feature point pair matching results and the image fusion algorithm.

[0017] Furthermore, deleting duplicate feature points in the comprehensive feature point set and screening the feature points in the comprehensive feature point set according to the local gradient of the feature point, the difference of the descriptor and the position of the feature point in the image to be stitched includes:

[0018] Calculating the descriptor similarity between each feature point, when the absolute value of the descriptor similarity difference between two feature points exceeds a first preset threshold, deleting one of the two feature points;

[0019] The quality score of each feature point is calculated based on the local gradient of each feature point in the comprehensive feature point set, the difference of the descriptor and the position of the feature point in the image to be stitched;

[0020] The feature points in the comprehensive feature point set whose quality scores are less than the second preset threshold are deleted.

[0021] Furthermore, the calculating of the quality score of each feature point according to the local gradient of each feature point in the comprehensive feature point set, the difference of the descriptor and the position of the feature point in the image to be stitched includes:

[0022] Calculate the pixel gradient of each feature point within a preset range; accumulate the gradient values ​​to obtain a quantitative index of the corresponding feature point, and calculate a first score of each feature point based on the quantitative index;

[0023] Calculating the similarity between the descriptor of each feature point and the descriptors of other feature points in the comprehensive feature point set; and calculating a second score for each feature point based on the similarity;

[0024] Determining whether the feature point is located in an edge area of ​​the image to obtain a third score for each feature point;

[0025] The first score, the second score, and the third score of each feature point are summed to obtain a quality score of each feature point.

[0026] Furthermore, the first feature extraction algorithm and the second feature extraction algorithm are run in parallel to simultaneously extract feature points in each of the images to be spliced.

[0027] Furthermore, the first feature extraction algorithm is a SIFT algorithm; and the second feature extraction algorithm is an ORB algorithm.

[0028] Furthermore, the image fusion algorithm is a weighted average method.

[0029] A third aspect of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the second aspect when executing the program.

[0030] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the second aspect.

[0031] Compared with the prior art, the present invention has the following advantages:

[0032] In the present invention, the closed panoramic display screen can display the content in 360 degrees without blind spots, and the audience can freely rotate the viewing angle to experience an immersive experience. This all-round display method helps the audience to understand the display object more comprehensively and enhances the interest and sense of participation in viewing. The closed panoramic display screen can present a realistic visual effect, making the audience feel as if they are in a real scene.

[0033] The present invention can make full use of the advantages of different algorithms and improve the accuracy and robustness of feature point extraction by combining the first feature extraction algorithm and the second feature extraction algorithm. Different feature extraction algorithms have different sensitivities to different features in the image, so the combined use can capture more comprehensive feature information. The comprehensive feature point set obtained at the same time contains feature points from the two algorithms, and the distribution and description of these feature points in the image are more comprehensive and detailed, providing a better foundation for subsequent feature point matching and image fusion.

[0034] By deleting duplicate feature points in the comprehensive feature point set, redundant calculations in the subsequent matching process can be reduced, and matching efficiency can be improved. Feature points can be screened based on the local gradient of feature points, the difference of descriptors, and the position of feature points in the image to be stitched, and those unstable or unqualified feature points can be further eliminated, and more representative and stable feature points can be retained, thereby improving the accuracy and reliability of matching.

[0035] Finally, the transformation model is used to match feature point pairs, which can fully consider the geometric transformation relationship between images and improve the matching accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0037] Figure 1 It is a schematic diagram of the overall structure of the present invention;

[0038] Figure 2 is a flow chart of the panoramic image stitching method of the present invention;

[0039] Figure 3 A schematic diagram of the structure of a computer system for implementing the VR-based panoramic image stitching method provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0041] In the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "upper", "lower", "inner", "back", etc. are based on the directions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0042] In addition, in the description of the present invention, unless otherwise clearly defined, the terms "installed", "connected", "connected", and "connector" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood in combination with specific circumstances.

[0043] The following will refer to the attached Figures 1 to 3 The present invention is described in detail with reference to embodiments.

[0044] Example 1

[0045] Overall, if Figure 1 As shown, the present invention provides a panoramic display system based on VR, comprising:

[0046] Closed panoramic display screen and video splicing server;

[0047] The video stitching server is used to stitch multiple panoramic images to merge and generate a panoramic video, and output the panoramic video to a closed panoramic display screen for panoramic display.

[0048] The closed panoramic display screen can display the content in 360 degrees without blind spots. The audience can freely rotate the viewing angle and feel the immersive experience. This all-round display method helps the audience to understand the display object more comprehensively and enhances the fun and participation of viewing. The closed panoramic display screen can present realistic visual effects, making the audience feel as if they are in the real scene.

[0049] In a possible implementation, the closed panoramic display screen includes a plurality of LED display screens;

[0050] Each LED display screen forms a closed polyhedron to display the panoramic video output by the video splicing server.

[0051] Example 2

[0052] like Figure 2 As shown, a VR-based panoramic image stitching method applied to Example 1 includes:

[0053] Step S1, obtaining a plurality of images to be stitched;

[0054] Step S2, extracting feature points in each image to be stitched based on the first feature extraction algorithm and the second feature extraction algorithm to obtain a comprehensive feature point set;

[0055] Step S3, deleting duplicate feature points in the comprehensive feature point set, and screening the feature points in the comprehensive feature point set according to the local gradient of the feature point, the difference of the descriptor and the position of the feature point in the image to be stitched;

[0056] Step S4, using a transformation model (such as homography matrix, affine transformation, etc.) to perform feature point pair matching;

[0057] Step S5: performing image fusion according to the feature point pair matching results and based on an image fusion algorithm to obtain a panoramic image.

[0058] The present invention can make full use of the advantages of different algorithms and improve the accuracy and robustness of feature point extraction by combining the first feature extraction algorithm and the second feature extraction algorithm. Different feature extraction algorithms have different sensitivities to different features in the image, so the combined use can capture more comprehensive feature information. The comprehensive feature point set obtained at the same time contains feature points from the two algorithms, and the distribution and description of these feature points in the image are more comprehensive and detailed, providing a better foundation for subsequent feature point matching and image fusion.

[0059] By deleting duplicate feature points in the comprehensive feature point set, redundant calculations in the subsequent matching process can be reduced, and matching efficiency can be improved. Feature points can be screened based on the local gradient of feature points, the difference of descriptors, and the position of feature points in the image to be stitched, and those unstable or unqualified feature points can be further eliminated, and more representative and stable feature points can be retained, thereby improving the accuracy and reliability of matching.

[0060] Finally, the transformation model is used to match feature point pairs, which can fully consider the geometric transformation relationship between images and improve the matching accuracy.

[0061] In a possible implementation, the method for obtaining a panoramic image by performing image fusion according to the matching results of feature points and based on an image fusion algorithm further includes performing color balancing and image brightness adjustment on the panoramic image.

[0062] In a possible implementation, repeated feature points in the comprehensive feature point set are deleted, and the feature points in the comprehensive feature point set are screened according to the local gradient of the feature point, the difference of the descriptor, and the position of the feature point in the image to be stitched, including:

[0063] Calculate the descriptor similarity between each feature point. When the absolute value of the descriptor similarity difference between two feature points exceeds a first preset threshold, the two feature points are considered to be duplicated, and one of the two feature points is deleted.

[0064] The quality score of each feature point is calculated based on the local gradient of each feature point in the comprehensive feature point set, the difference of the descriptor and the position of the feature point in the image to be stitched;

[0065] The feature points in the comprehensive feature point set whose quality scores are less than the second preset threshold are deleted.

[0066] In one possible implementation, the quality score of each feature point is calculated based on the local gradient of each feature point in the comprehensive feature point set, the difference of the descriptor, and the position of the feature point in the image to be stitched, including:

[0067] The pixel gradient of each feature point within a preset range is calculated by an edge detection (Sobel Operator, Sobel) operator or a cross differential operator (Prewitt Operator, Prewitt) operator; the gradient values ​​are accumulated to obtain a quantitative index of the corresponding feature point, and the higher the quantitative index, the higher the gradient value, indicating that the texture around the feature point is richer and the stability is higher. The first score of each feature point is calculated based on the quantitative index; in this embodiment, multiple thresholds can be set, and different first scores correspond to different threshold intervals of the quantitative index, wherein the larger the value of the quantitative index, the higher the score.

[0068] The similarity between the descriptor of each feature point and the descriptors of other feature points in the comprehensive feature point set is calculated by using the Euclidean distance algorithm or the Hamming distance algorithm; if the descriptor of a feature point is very similar to multiple other descriptors, it means that the uniqueness of the descriptor is poor and it may be a redundant or misdetected feature point; the second score of each feature point is calculated based on the similarity; in this embodiment, multiple similarity thresholds can be set, and different second scores correspond to the similarities between the descriptor of a feature point and multiple other descriptors in different threshold intervals. The higher the similarity, the lower the second score.

[0069] Determine whether the feature point is located in the edge area of ​​the image to obtain the third score of each feature point; feature points in the edge area of ​​the image (such as the sky, wall, etc.) may produce large errors when stitching due to image cropping or distortion. The feature points in these areas may not be unique enough and are easily confused with feature points in other areas. Prioritize feature points located in different and representative areas of the image, such as corner points of buildings, outline points of trees, etc.

[0070] The first score, the second score, and the third score of each feature point are summed to obtain a quality score of each feature point.

[0071] In a possible implementation, the first feature extraction algorithm and the second feature extraction algorithm are run in parallel to simultaneously extract feature points in each image to be spliced, thereby improving the efficiency of image processing.

[0072] In a possible implementation, the first feature extraction algorithm is a Scale Invariant Feature Transform (SIFT) algorithm; the second feature extraction algorithm is an ORB (Oriented FAST and Rotated BRIEF) algorithm.

[0073] The SIFT algorithm is a feature point detection algorithm based on scale space. It can extract stable feature points at different scales and rotation angles and generate unique descriptors. However, the SIFT algorithm has poor detection effect on smooth edge areas. This is mainly because these areas lack sufficient texture and detail information, making it difficult for the SIFT algorithm to accurately locate feature points in these areas. The ORB (Oriented FAST and Rotated BRIEF) algorithm is an efficient feature point detection algorithm that combines the FAST corner detection algorithm and the BRIEF descriptor algorithm. When detecting feature points, the ORB algorithm mainly relies on the FAST algorithm, which quickly locates corner points by comparing the brightness difference between pixels and their surrounding pixels. Since the FAST algorithm has low requirements for the texture and detail information of the image, it can detect more feature points in smooth edge areas. In addition, the ORB algorithm also achieves rotation invariance by introducing orientation information and rotating the BRIEF descriptor, thereby improving the accuracy and robustness of feature matching.

[0074] In summary, the ORB algorithm is used to solve the problem that the SIFT algorithm has poor detection effect in smooth edge areas.

[0075] In a possible implementation, the image fusion algorithm is a weighted average method.

[0076] The weighted average method directly processes the pixels of the image. Its calculation process is relatively simple and the amount of calculation is small, which makes the algorithm have high application value in real-time processing or resource-constrained environments. At the same time, since the algorithm is relatively direct, it is also relatively easy to implement in programming. By setting the weights reasonably, the weighted average method can improve the signal-to-noise ratio of the fused image to a certain extent, making the fused image clearer and more accurate. This advantage is particularly important in the field of image processing, because the signal-to-noise ratio of the image directly affects the quality of the image and the effect of subsequent processing. At the same time, the weighted average method allows the selection of appropriate weights according to different application requirements to obtain the best fusion effect. This flexibility enables the algorithm to adapt to different image fusion scenarios and meet the needs of different users. Finally, there are fewer false pixels: the weighted average method directly performs weighted average processing on the image pixels, reducing the generation of false pixels and making the fused image information more accurate and reliable. Since the calculation process of the weighted average method is relatively simple, the algorithm is easier to implement on hardware, which is conducive to improving the efficiency and speed of image processing.

[0077] In a specific implementation, the weighted average method is used to fuse the pixels in the overlapping area of ​​the images. The pixels are weighted and then averaged. The corresponding weights are set and multiplied with the pixels in the overlapping area of ​​the images to be spliced ​​and the results are accumulated to form a new image. In this way, the grayscale values ​​of the overlapping area can be smoothly transitioned to eliminate the gaps in the images to be spliced. The weights can be selected using a fade-in and fade-out algorithm.

[0078] According to the formula

[0079]

[0080] Calculate the pixel value f(a,b) corresponding to the pixel point (a,b) after fusion; where h1(a,b) and h2(a,b) are the pixel values ​​corresponding to the first image to be stitched and the second image to be stitched at the pixel point (a,b), respectively; k1(a,b) and k2(a,b) are the weights of the overlapping parts of h1(a,b) and h2(a,b), respectively.

[0081] like Figure 3 As shown, a computer system suitable for implementing the panoramic image stitching method provided in the above embodiment includes a central processing module (CPU), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage part into a random access memory (RAM). In the RAM, various programs and data required for the operation of the computer system are also stored. The CPU, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0082] The following components are connected to the I / O interface: an input section including a keyboard, a mouse, etc.; an output section including a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section including a hard disk, etc.; and a communication section including a network interface card such as a LAN card, a modem, etc. The communication section performs communication processing via a network such as the Internet. A drive is also connected to the I / O interface as needed. Removable media, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., are installed on the drive as needed so that a computer program read therefrom is installed into the storage section as needed.

[0083] In particular, according to this embodiment, the process described in the flowchart above can be implemented as a computer software program. For example, this embodiment includes a computer program product, which includes a computer program tangibly contained on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part, and / or installed from a removable medium.

[0084] The flow chart and schematic diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product of the present embodiment. In this regard, each square box in the flow chart or schematic diagram can represent a part of a module, program segment or code, and a part of the above-mentioned module, program segment or code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the schematic diagram and / or flow chart, and the combination of the square boxes in the schematic diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0085] As another aspect, the present embodiment further provides a non-volatile computer storage medium, which may be a non-volatile computer storage medium included in the above-mentioned device in the above-mentioned embodiment, or a non-volatile computer storage medium that exists independently and is not installed in a terminal. The above-mentioned non-volatile computer storage medium stores one or more programs, and when the above-mentioned one or more programs are executed by a device, the above-mentioned device: step S1, obtain multiple images to be stitched; step S2, extract feature points in each image to be stitched based on the first feature extraction algorithm and the second feature extraction algorithm to obtain a comprehensive feature point set; step S3, delete repeated feature points in the comprehensive feature point set, and screen the feature points in the comprehensive feature point set according to the local gradient of the feature point, the difference of the descriptor and the position of the feature point in the image to be stitched; step S4, use a transformation model (such as a homography matrix, an affine transformation, etc.) to match feature point pairs; step S5, perform image fusion based on the feature point pair matching results and the image fusion algorithm to obtain a panoramic image.

[0086] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "upper" and "lower" and the like is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. Unless otherwise clearly specified and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be a connection between the two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

[0087] It should also be noted that, in the description of the present invention, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0088] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not limitations on the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is impossible to list all the embodiments here. All obvious changes or modifications derived from the technical solution of the present invention are still within the protection scope of the present invention.

Claims

1. A panoramic display system based on VR, characterized in that: include: Closed panoramic display screen and video splicing server; The video stitching server is used to stitch multiple panoramic images to merge and generate a panoramic video, and output the panoramic video to a closed panoramic display screen for panoramic display.

2. A VR-based panoramic display system according to claim 1, characterized in that: The closed panoramic display screen includes a plurality of LED display screens; Each of the LED display screens forms a closed polyhedron to perform a panoramic display of the panoramic video output by the video splicing server.

3. A VR-based panoramic image stitching method applied to a VR-based panoramic display system as described in any one of claims 1-2, characterized in that: include: Acquire multiple images to be stitched; Based on the first feature extraction algorithm and the second feature extraction algorithm, feature points in each image to be stitched are respectively extracted to obtain a comprehensive feature point set; Delete duplicate feature points in the comprehensive feature point set, and screen the feature points in the comprehensive feature point set according to the local gradient of the feature point, the difference of the descriptor and the position of the feature point in the image to be stitched; Use transformation model to match feature point pairs; The panoramic image is obtained by performing image fusion based on the feature point pair matching results and the image fusion algorithm.

4. The VR-based panoramic image stitching method according to claim 3, characterized in that: The deleting of duplicate feature points in the comprehensive feature point set and screening the feature points in the comprehensive feature point set according to the local gradient of the feature point, the difference of the descriptor and the position of the feature point in the image to be stitched includes: Calculating the descriptor similarity between each feature point, when the absolute value of the descriptor similarity difference between two feature points exceeds a first preset threshold, deleting one of the two feature points; The quality score of each feature point is calculated based on the local gradient of each feature point in the comprehensive feature point set, the difference of the descriptor and the position of the feature point in the image to be stitched; The feature points in the comprehensive feature point set whose quality scores are less than the second preset threshold are deleted.

5. The VR-based panoramic image stitching method according to claim 4, characterized in that: The step of calculating the quality score of each feature point according to the local gradient of each feature point in the comprehensive feature point set, the difference of the descriptor and the position of the feature point in the image to be stitched comprises: Calculate the pixel gradient of each feature point within a preset range; accumulate the gradient values ​​to obtain a quantitative index of the corresponding feature point, and calculate a first score of each feature point based on the quantitative index; Calculating the similarity between the descriptor of each feature point and the descriptors of other feature points in the comprehensive feature point set; and calculating a second score for each feature point based on the similarity; Determining whether the feature point is located in an edge area of ​​the image to obtain a third score for each feature point; The first score, the second score, and the third score of each feature point are summed to obtain a quality score of each feature point.

6. The VR-based panoramic image stitching method according to claim 3, characterized in that: The first feature extraction algorithm and the second feature extraction algorithm are run in parallel to simultaneously extract feature points in each of the images to be spliced.

7. The VR-based panoramic image stitching method according to claim 3, characterized in that: The first feature extraction algorithm is a SIFT algorithm; the second feature extraction algorithm is an ORB algorithm.

8. The VR-based panoramic image stitching method according to claim 3, characterized in that: The image fusion algorithm is a weighted average method.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 3 to 8 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 3 to 8 is implemented.

Images