Depth map processing method and apparatus, and storage medium

Abstract

The embodiment of the present disclosure provides a depth map processing method, device and storage medium. The depth map and the color map matched with the depth map are obtained, target detection is performed on the color map, the color map is divided into regions based on the contour of the detected target, a contour region is obtained, a hollow region is extracted from the depth map, and the depth map and the color map after region division are intersected based on the matching relationship between the depth map and the color map, so as to obtain a sub-region intersected with the contour region in the hollow region. The sub-region is the sub-region of the hollow region and the contour region, and the sub-region is the sub-region of the sub-region. The sub-region can be effectively distinguished according to the sub-region. Therefore, the embodiment can accurately distinguish the sub-region of the hollow region, and the depth value of the sub-region can be accurately determined and corresponding processing such as filling can be performed.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to depth map processing methods, apparatus and storage media. Background Technology

[0002] In 3D computer graphics and computer vision, a depth map is an image or image channel that contains information about the distance of the surface of a scene object from the viewpoint, used to simulate 3D shapes or reconstruct them.

[0003] However, when acquiring depth maps, areas may appear that the camera cannot capture, resulting in holes at the edges of the depth map, such as... Figure 1 As shown, the white hand area 10 has a hollow area 11 on its right edge. This hollow area 11 includes both the arm and the ceiling, but their depth values ​​differ significantly. If the depth values ​​of the effective neighboring points around the hollow are used directly for filling, there will be a large error for some points within the hollow. For example, the depth value of the arm might become the depth value of the ceiling, and vice versa. Therefore, without effectively distinguishing the hollow areas, accurate depth information cannot be filled.

[0004] Therefore, how to accurately distinguish the hole regions in the depth map is a common issue that the industry is considering. Summary of the Invention

[0005] In view of the shortcomings of the above-mentioned related technologies, the purpose of this disclosure is to provide a depth map processing method, apparatus and storage medium to solve the technical problem that the related technologies cannot accurately distinguish the hole regions in the depth map.

[0006] The first aspect of this disclosure provides a depth map processing method, which includes:

[0007] Obtain a depth map and a color map registered with the depth map;

[0008] Target detection is performed on the color image, and the color image is divided into regions based on the contours of the detected targets to obtain contour regions;

[0009] Hole regions are extracted from the depth map. Based on the registration relationship between the depth map and the color map, the intersection of the depth map and the color map with region division is calculated to obtain the sub-regions in the hole regions that intersect with the contour regions.

[0010] In some embodiments, target detection is performed on the color image, and the color image is divided into regions based on the contours of the detected targets to obtain contour regions, including:

[0011] Image semantic segmentation is performed on the color image to obtain the contour region.

[0012] In some embodiments, image semantic segmentation is performed on the color image to obtain contour regions, including:

[0013] The Mask R-CNN model is used to perform semantic segmentation on the color image to obtain the contour region.

[0014] In some embodiments, the intersection of the depth map and the color map used for region segmentation is calculated to obtain sub-regions in the hole region that intersect with the contour region, including:

[0015] When multiple contour regions are obtained from the color image, the intersection of the depth image and the color image used for region division is calculated to obtain a sub-region where the hole region intersects with at least two contour regions.

[0016] In some embodiments, the depth map processing method further includes:

[0017] The sub-region is filled with depth pixels based on the depth information of the effective neighboring points of the sub-region.

[0018] A second aspect of this disclosure also provides a depth map processing apparatus, comprising:

[0019] The acquisition module acquires the depth map and a color image registered with the depth map;

[0020] The image segmentation module performs target detection on the color image and divides the color image into regions based on the contours of the detected targets to obtain contour regions.

[0021] The intersection module extracts the hole region from the depth map. Based on the registration relationship between the depth map and the color map, it calculates the intersection of the depth map and the color map that has been divided into regions to obtain the sub-regions in the hole region that intersect with the contour region.

[0022] In some embodiments, the image segmentation module is specifically used for:

[0023] Image semantic segmentation is performed on the color image to obtain the contour region.

[0024] In some embodiments, the depth map processing apparatus further includes:

[0025] The filling module fills the sub-region with depth pixels based on the depth information of the effective neighboring points of the sub-region.

[0026] The fifth aspect of this disclosure provides a computer apparatus, including: a communicator, a memory, and a processor; the communicator is used to communicate with an external source; the memory stores program instructions; and the processor is used to execute the program instructions to perform a depth map processing method as described in any of the first aspects.

[0027] The fourth aspect of this disclosure provides a computer-readable storage medium storing program instructions that are executed to perform a depth map processing method as described in any of the first aspects.

[0028] As described above, this disclosure provides a depth map processing method, apparatus, and storage medium. By acquiring a depth map and a color image registered with the depth map, target detection is performed on the color image. Based on the contours of the detected targets, the color image is divided into regions to obtain contour regions. Hole regions are extracted from the depth map. Based on the registration relationship between the depth map and the color image, the intersection of the depth map and the region-divided color image is calculated to obtain sub-regions within the hole regions that intersect with the contour regions. These sub-regions are sub-regions of both the hole regions and the contour regions. Therefore, sub-regions are sub-regions of sub-regions, and can be effectively distinguished. Thus, this embodiment can accurately distinguish sub-regions of the hole region. Under this premise, the depth value of the sub-region can be accurately determined and corresponding processing such as filling can be performed. Attached Figure Description

[0029] Figure 1 A depth map that demonstrates the relevant technologies.

[0030] Figure 2 One of the flowcharts illustrating a depth map processing method according to an embodiment of the present disclosure.

[0031] Figure 3 This diagram illustrates a region segmentation obtained by performing target detection on a color image according to an embodiment of the present disclosure.

[0032] Figure 4 A schematic diagram illustrating the principle of the depth map processing method according to an embodiment of the present disclosure.

[0033] Figure 5 A second flowchart illustrating the depth map processing method according to an embodiment of this disclosure.

[0034] Figure 6 One of the schematic diagrams of a depth map processing apparatus according to an embodiment of the present disclosure is shown.

[0035] Figure 7 A second schematic diagram of a depth map processing apparatus according to an embodiment of the present disclosure.

[0036] Figure 8 A schematic diagram of the structure of a computer device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0037] The following specific examples illustrate the implementation of this disclosure. Those skilled in the art can easily understand other advantages and effects of this disclosure from the information disclosed herein. This disclosure can also be implemented or applied to systems through other different specific embodiments, and various details in this disclosure can also be modified or changed according to different viewpoints and application systems without departing from the spirit of this disclosure. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this disclosure can be combined with each other.

[0038] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings, so that those skilled in the art to which this disclosure pertains can readily implement it. This disclosure may be embodied in many different forms and is not limited to the embodiments described herein.

[0039] In this disclosure, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic represented in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. Furthermore, the specific features, structures, materials, or characteristics represented may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples represented in this disclosure, as well as the features of those different embodiments or examples.

[0040] Furthermore, the terms "first" and "second" are used for illustrative purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the representation of this disclosure, "a plurality of" means two or more, unless otherwise expressly and specifically defined.

[0041] For the purpose of clarity, devices unrelated to the description are omitted, and the same or similar components throughout the specification are given the same reference numerals.

[0042] Throughout this specification, when it is said that a device is "connected" to another device, this includes not only "direct connection" but also "indirect connection" by placing other components in between. Furthermore, when it is said that a device "comprises" a certain constituent element, unless otherwise stated otherwise, this does not exclude other constituent elements, but rather implies that other constituent elements may be included.

[0043] Although the terms first, second, etc., are used in some instances herein to refer to various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, first interface and second interface, etc., are used. Furthermore, as used herein, the singular forms “a,” “an,” and “the” are intended to also include the plural forms unless the context indicates otherwise. It should be further understood that the terms “comprising,” “including,” indicate the presence of features, steps, operations, elements, modules, items, kinds, and / or groups, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, modules, items, kinds, and / or groups. The terms “or” and “and / or” as used herein are to be interpreted as inclusive, or mean any one or any combination thereof. Thus, “A, B, or C” or “A, B, and / or C” means “any one of: A; B; C; A and B; A and C; B and C; A, B, and C.” An exception to this definition will only occur if the combination of elements, functions, steps, or operations is inherently mutually exclusive in some way.

[0044] The technical terms used herein are for reference only to specific embodiments and are not intended to limit the scope of this disclosure. The singular form used herein includes the plural form unless the statement explicitly indicates otherwise. The word "comprising" as used in this specification means to specify a particular characteristic, region, integer, step, action, element, and / or component, and does not exclude the presence or addition of other characteristics, regions, integers, steps, actions, elements, and / or components.

[0045] Terms such as "below" and "above" indicating relative space are used to more easily explain the relationship of one device relative to another in the accompanying drawings. These terms refer not only to their meaning as shown in the drawings but also to other meanings or functions of the device in use. For example, if the device in the drawings is rotated, a device previously described as "below" another device may now be described as "above" another device. Therefore, the exemplary term "below" encompasses both above and below. The device may be rotated 90° or other angles, and the terms representing relative space are interpreted accordingly.

[0046] Although not explicitly defined, all terms, including technical and scientific terms used herein, shall have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. Terms defined in commonly used dictionaries shall be further interpreted as having a meaning consistent with the relevant technical literature and the message of the present disclosure, and shall not be over-interpreted as having an ideal or overly formulaic meaning unless otherwise defined.

[0047] Figure 2 This is a flowchart of a depth map processing method provided in an embodiment of this disclosure. The execution subject of this method is a depth map processing system, such as... Figure 2 As shown, this method includes the following steps:

[0048] Step 210: Obtain the depth map and the color map registered with the depth map;

[0049] Step 220: Perform target detection on the color image, and divide the color image into regions based on the contours of the detected targets to obtain contour regions;

[0050] Step 230: Extract the hole region from the depth map. Based on the registration relationship between the depth map and the color map, find the intersection of the depth map and the color map that has been divided into regions to obtain the sub-regions in the hole region that intersect with the contour region.

[0051] In this context, a sub-region refers to a sub-region of both the hole region and the outline region. Since a sub-region is a sub-region of a sub-region, it can be effectively distinguished. Therefore, this embodiment can accurately distinguish sub-regions within a hole region. Under this premise, the depth value of the sub-region can be accurately determined, and corresponding processing such as filling can be performed.

[0052] In this embodiment, the depth map and the color map are registered, which includes that they are the same size and that the positions of each target are consistent, which improves the feasibility of the solution in this embodiment.

[0053] In this embodiment of the disclosure, target detection is performed on the color image to detect various targets, contour regions, and positions in the color image.

[0054] In this embodiment of the disclosure, target detection is performed on a color image, and the color image is divided into regions based on the detected target contours to obtain contour regions, including:

[0055] Image semantic segmentation is performed on the color image to obtain the contour region.

[0056] Image semantic segmentation is a classification process at the pixel level. Pixels belonging to the same category are grouped together. Therefore, semantic segmentation understands images at the pixel level. For example, in the following photo, pixels belonging to people are grouped into one category, pixels belonging to motorcycles are grouped into another, and background pixels are also grouped into a separate category.

[0057] In this embodiment of the disclosure, specifically, the Mask R-CNN model can be used to perform semantic segmentation on the color image to obtain the contour region.

[0058] Mask R-CNN is an extension of Faster R-CNN that can effectively detect objects in images. Mask R-CNN is easy to train, requiring only a small overhead to be added to Faster R-CNN, while also generating a high-quality segmentation mask for each instance.

[0059] In this embodiment, the color image is input into the Mask R-CNN model to segment the color image based on contour regions, thereby obtaining multiple contour regions from the color image.

[0060] like Figure 3 As shown, after dividing the color map into regions, various types of cars, pedestrians, roads, and buildings can be seen. Figure 3 The image is displayed as a grayscale image, but in practical applications, there are differences in the pixel values ​​of each target, so different colors are used to display them.

[0061] In this embodiment, the Mask R-CNN model is a semantic segmentation algorithm. Other deep learning models based on convolutional neural networks or other non-deep learning models can also be used to perform semantic segmentation on color images, which is not limited here.

[0062] In this embodiment of the disclosure, the intersection of the depth map and the color map used for region division is determined based on the positional registration relationship. Therefore, a hole region can intersect with at least one contour region, thereby obtaining at least one sub-region from the hole region.

[0063] In one case, if a void region intersects with multiple contour regions, then the void region is divided into multiple sub-regions, and the edges of two adjacent sub-regions are the boundaries of the corresponding contour regions.

[0064] In this way, when multiple contour regions are obtained from the color image, the intersection of the depth image and the color image used for region division is calculated to obtain a sub-region where the hole region intersects with at least two contour regions.

[0065] like Figure 4 As shown, the cavity region H1 is displayed in the depth map D, and the color map R registered with the depth map D is divided into four contour regions R1, R2, R3 and R4.

[0066] In the case of finding the intersection, the hole region H1 includes the sub-region I1 that intersects with the contour region R2 and the sub-region I2 that intersects with the contour region R3.

[0067] Figure 5 A flowchart illustrating a depth map processing method according to an embodiment of the present disclosure is provided. The method specifically includes the following steps:

[0068] Step 510: Obtain the depth map and the color map registered with the depth map;

[0069] Step 520: Perform target detection on the color image, and divide the color image into regions based on the contours of the detected targets to obtain contour regions;

[0070] Step 530: Extract the hole region from the depth map. Based on the registration relationship between the depth map and the color map, find the intersection of the depth map and the color map that has been divided into regions to obtain the sub-regions in the hole region that intersect with the contour region.

[0071] Step 540: Fill the sub-region with depth pixels based on the depth information of the effective neighboring points of the sub-region.

[0072] In this embodiment, the performance of the hole-filling algorithm is improved by segmenting the color image corresponding to the depth map. That is, each sub-region in the hole is filled only with the effective depth value from the same target, thus avoiding large errors.

[0073] This disclosure also provides a specific depth map processing method, which specifically includes the following steps:

[0074] First, obtain the registered depth map D and color map R, such as... Figure 4 As shown;

[0075] Next, the color image R is segmented at the pixel level, and each image is divided into multiple regions R1…Rn after segmentation;

[0076] Next, the cavity regions H1…Hm are extracted from the depth map D;

[0077] Next, find the intersection of multiple regions Rn and the hole region Hm obtained from the color image R to obtain the region intersection I1..Ik. It can be seen that each sub-region Ii in I can find its corresponding parent region Rn in the color image R.

[0078] Finally, the sub-regions are filled with depth pixels based on the depth pixels of their neighboring points.

[0079] like Figure 6 The diagram illustrates a module schematic of a depth map processing apparatus according to an embodiment of this disclosure. It should be noted that the principle of the depth map processing apparatus can be referenced from the depth map processing methods in previous embodiments; therefore, the same technical content will not be repeated here.

[0080] The depth map processing device 600 may include:

[0081] The acquisition module 610 acquires the depth map and the color map registered with the depth map;

[0082] The image segmentation module 620 performs target detection on the color image and divides the color image into regions based on the contours of the detected targets to obtain contour regions;

[0083] The intersection module 630 extracts the hole region from the depth map. Based on the registration relationship between the depth map and the color map, it calculates the intersection of the depth map and the color map that has been divided into regions to obtain the sub-regions in the hole region that intersect with the contour region.

[0084] In some embodiments, the image segmentation module 620 is specifically used for:

[0085] Image semantic segmentation is performed on the color image to obtain the contour region.

[0086] In some embodiments, with Figure 6 compared to, Figure 7 The depth map processing device 700 shown also includes:

[0087] The filling module 710 fills the sub-region with depth pixels based on the depth information of the effective neighboring points of the sub-region.

[0088] It should be noted that, in Figure 6 and 7 The various functional modules in the embodiments can be implemented, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, they can be implemented, in whole or in part, in the form of a program instruction product. A program instruction product includes one or more program instructions. When the program instructions are loaded and executed on a computer, all or part of the flow or function according to this disclosure is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The program instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0089] and, Figure 6 and 7 The apparatus disclosed in the embodiments can be implemented through other modular division methods. The apparatus embodiments shown above are merely illustrative. For example, the module division is only a logical functional division. In actual implementation, there may be other division methods. For example, multiple modules or modules may be combined or dynamically integrated into another system, or some features may be ignored or not executed. Furthermore, the shown or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces. The indirect coupling or communication connection between devices or modules may be electrical or other forms.

[0090] in addition, Figure 6 and 7The functional modules and sub-modules in the embodiments can be dynamically integrated within a single processing unit, or each module can exist physically independently, or two or more modules can be dynamically integrated within a single unit. These dynamic units can be implemented in hardware or as software functional modules. If these dynamic units are implemented as software functional modules and executed as independent products for sale or use, they can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a hard disk, or an optical disk, etc.

[0091] This disclosure also provides a computer-readable storage medium storing program instructions, which are executed before being run. Figure 2 or Figure 5 The method steps in the embodiments.

[0092] The method steps in the above embodiments are implemented as software or computer code that can be stored in a recording medium (such as a CD ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or implemented as computer code that is originally stored in a remote recording medium or a non-transitory machine-readable medium and will be stored in a local recording medium after being downloaded via a network. Thus, the method represented herein can be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA).

[0093] It should be specifically noted that the flowchart representations of the embodiments described above in this disclosure can be understood as representing modules, segments, or portions of code comprising one or more executable instructions for implementing a specific logical function or process. Furthermore, the scope of the preferred embodiments of this disclosure includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved.

[0094] For example, Figure 2 The order of the steps in the embodiment shown in 5 may vary in specific scenarios and is not limited to the above representation.

[0095] like Figure 8 The diagram shown illustrates the structure of a computer device according to an embodiment of the present disclosure.

[0096] In some embodiments, the computer device is used to load program instructions for implementing a depth map processing method. The computer device may specifically be implemented as, for example, a server, desktop computer, laptop computer, mobile terminal, etc., and may be used by an implementer who stores and / or runs these program instructions for commercial purposes such as development or testing.

[0097] Figure 8The computer device 800 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0098] like Figure 8 As shown, the computer device 800 is presented in the form of a general-purpose computing device. The components of the computer device 800 may include, but are not limited to: at least one processing unit 810, at least one storage unit 820, and a bus 830 connecting different system components (including storage unit 820 and processing unit 810).

[0099] The storage unit stores program code, which can be executed by the processing unit 810, enabling the computer device to implement the embodiments of the present disclosure (e.g., Figure 2 The method steps are described in the example.

[0100] In some embodiments, storage unit 820 may include volatile storage units, such as random access memory (RAM) 821 and / or cache storage unit 822, and may further include read-only memory (ROM) 823.

[0101] In some embodiments, the storage unit 820 may also include a program / utility 824 having a set (at least one) of program modules 825, such program modules 825 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0102] In some embodiments, bus 830 may include a data bus, an address bus, and a control bus.

[0103] In some embodiments, the computer device 800 may also communicate with one or more external devices 80 (e.g., keyboard, pointing device, Bluetooth device, etc.) via an input / output (I / O) interface 850. Optionally, the computer device 800 also includes a display unit 840 connected to the input / output (I / O) interface 850 for display purposes. Furthermore, the computer device 800 may also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via a network adapter 860. As shown, the network adapter 860 communicates with other modules of the computer device 800 via a bus 830. It should be understood that, although not shown in the figures, other hardware and / or software modules may be used in conjunction with the computer device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0104] In summary, the embodiments of this disclosure provide a depth map processing method, apparatus, and storage medium. By acquiring a depth map and a color image registered with the depth map, target detection is performed on the color image, and the color image is divided into regions based on the contours of the detected targets to obtain contour regions. Hole regions are extracted from the depth map. Based on the registration relationship between the depth map and the color image, the intersection of the depth map and the region-divided color image is calculated to obtain sub-regions within the hole regions that intersect with the contour regions. These sub-regions are sub-regions of both the hole regions and the contour regions; therefore, sub-regions are sub-regions of sub-regions, and can be effectively distinguished. Thus, this embodiment can accurately distinguish sub-regions of the hole region, and under this premise, the depth value of the sub-region can be accurately determined and corresponding processing such as filling can be performed.

[0105] The above embodiments are merely illustrative of the principles and effects of this disclosure and are not intended to limit this disclosure. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this disclosure. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this disclosure should still be covered by the claims of this disclosure.

Claims

1. A method of depth map processing, characterized by, The method comprises the following steps: obtaining a depth map and a color map registered with the depth map; performing target detection on the color map, and performing region division on the color map based on the contour of the detected target to obtain a contour region; extracting a hollow region from the depth map, and performing intersection between the depth map and the color map subjected to region division based on the registration relationship between the depth map and the color map to obtain a sub-region intersected with the contour region in the hollow region; performing depth pixel filling on the sub-region based on the depth information of effective neighborhood points of the sub-region, the effective neighborhood points being from the same target corresponding to the sub-region and having effective depth values.

2. The depth map processing method of claim 1, wherein, The step of performing target detection on the color map, and performing region division on the color map based on the contour of the detected target to obtain a contour region comprises the following steps: performing image semantic segmentation on the color map to obtain a contour region.

3. The depth map processing method of claim 2, wherein, The step of performing image semantic segmentation on the color map to obtain a contour region comprises the following step: performing image semantic segmentation on the color map by using a Mask R-CNN model to obtain a contour region.

4. The depth map processing method of claim 1, wherein, The step of performing intersection between the depth map and the color map subjected to region division to obtain a sub-region intersected with the contour region in the hollow region comprises the following step: in the case where a plurality of contour regions are obtained from the color map, performing intersection between the depth map and the color map subjected to region division to obtain a sub-region intersected with at least two contour regions in the hollow region.

5. A depth map processing apparatus, characterized by, The method comprises the following steps: an obtaining module, configured to obtain a depth map and a color map registered with the depth map; an image division module, configured to perform target detection on the color map, and perform region division on the color map based on the contour of the detected target to obtain a contour region; an intersection module, configured to extract a hollow region from the depth map, and perform intersection between the depth map and the color map subjected to region division based on the registration relationship between the depth map and the color map to obtain a sub-region intersected with the contour region in the hollow region; a filling module, configured to perform depth pixel filling on the sub-region based on the depth information of effective neighborhood points of the sub-region, the effective neighborhood points being from the same target corresponding to the sub-region and having effective depth values.

6. The depth map processing apparatus according to claim 5, wherein, The image division module is specifically configured to: perform image semantic segmentation on the color map to obtain a contour region.

7. A computer apparatus, comprising: The device comprises a communicator, a memory and a processor; the communicator is configured to communicate with the outside; the memory stores program instructions; and the processor is configured to run the program instructions to perform the depth map processing method according to any one of claims 1 to 4. The device stores program instructions, which are run to perform the depth map processing method according to any one of claims 1 to 4.

8. A computer-readable storage medium, characterized in that, ​

Images