Pulse condition prior fused double-mask multi-mode tongue image analysis method and system

Abstract

The invention discloses a double-mask multi-mode tongue picture analysis method and system fusing pulse condition prior. The method comprises the following steps: firstly, constructing an interactive double-flow trunk based on a convolutional neural network and Transform, and extracting multi-scale visual features; secondly, designing a double-mask feature decoupling module, filtering a background by using an inner mask to purify internal textures, and constructing an edge enhancement branch by using a large-scale expanded outer mask in cooperation with a Scharr operator to accurately capture high-frequency features such as tooth marks and cracks; then, constructing a pulse condition guiding module, introducing a numerical pulse condition vector through a FiLM mechanism, and performing channel-level dynamic modulation on the visual features; and finally, carrying out model optimization by adopting two-stage deep supervision course learning and a cosine annealing hot restart strategy. According to the method, the conflict between the edge features and internal texture extraction is effectively solved, the limitation of a single visual mode is overcome, and the recognition precision of the subtle morphological features of the tongue image is improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a method for tongue image segmentation and classification based on a hybrid architecture of convolutional neural networks and Transformers. Background Technology

[0002] Limitations of Traditional Chinese Medicine Diagnosis: Tongue examination, as a core component of "inspection" in Traditional Chinese Medicine, is a crucial basis for assessing the state of the body's internal organs and Qi and blood. For a long time, tongue examination has primarily relied on the physician's visual observation and clinical experience. However, this model is inherently subjective and uncertain, for example:

[0003] (1) Non-standardization: Different doctors may have cognitive biases in describing the same tongue appearance, such as the description of "pale red" and "red".

[0004] (2) Environmental dependence: The intensity of light and changes in color temperature will affect the physician's judgment of tongue color and coating color.

[0005] (3) Difficulty in inheritance: The identification of subtle abnormal morphological features often relies on the physician's experience accumulation, which is difficult to quantify and pass on.

[0006] Current technological status: Computer-aided tongue imaging technology is an important research direction for objective diagnosis in Traditional Chinese Medicine. Currently, mainstream automated tongue imaging methods are mainly based on deep convolutional neural networks (CNN) or visual Transformer (ViT) architectures to perform semantic segmentation and classification on the acquired tongue images.

[0007] However, existing technologies still have significant limitations when dealing with complex and unusual morphological features. First, there is a conflict in spatial constraints between tongue segmentation and feature extraction. The "teeth marks" feature in tongue images manifests as indentations at the edges of the tongue, which are high-frequency edge features. Traditional semantic segmentation networks tend to generate smooth masks that closely adhere to the tongue body, causing the teeth marks located at the edge indentations to be filtered out as background. If the mask range is simply expanded to preserve the teeth marks, background noise such as lips, teeth, and oral shadows will be introduced into the feature extraction area, interfering with subsequent classification and judgment of tongue color and tongue coating color.

[0008] Secondly, the downsampling mechanism of deep neural networks leads to the loss of fine textures. In order to obtain global semantic information, existing models typically downsample feature maps multiple times. Although this process preserves the macroscopic structure of the image, it filters out high-frequency spatial details such as fine cracks and punctures, resulting in a decrease in the model's sensitivity to the detection of small pathological targets.

[0009] Furthermore, a single visual modality struggles to distinguish between visually similar syndromes that differ in underlying pathology. In Traditional Chinese Medicine (TCM) diagnosis, complex syndromes such as "true cold with false heat" exist, and relying solely on image features can easily lead to misdiagnosis. Existing multimodal fusion methods often involve simple decision-level scoring and fusion, lacking a mechanism for dynamically weighting and correcting visual feature channels using non-visual data (such as pulse diagnosis) during the feature extraction stage. This prevents the effective utilization of prior physiological knowledge to improve classification accuracy. Summary of the Invention

[0010] To address the aforementioned technical problems, this invention provides a dual-mask multimodal tongue image analysis method that integrates pulse priors, mainly comprising the following steps:

[0011] First, data acquisition and vectorization. The tongue image of the object to be detected and its corresponding numerical pulse feature vector are acquired. The pulse feature vector is obtained by extracting from the pulse description text and mapping it to a predefined multi-dimensional orthogonal feature space.

[0012] Second, the dual-stream feature extraction step. A multimodal fusion backbone network containing convolutional neural network branches and Transformer branches is constructed. The convolutional neural network branches are used to extract shallow high-resolution features and mid-level semantic features of the tongue image, while the Transformer branches are used to extract global contextual features based on a feature injection mechanism.

[0013] Third, the double-mask feature decoupling step. Based on the segmentation probability map output by the network. Construct inner masks respectively outer mask Specifically, this refers to: probability graphs After binarization, morphological erosion is performed using structuring elements to generate... Used to filter out background noise and lock the internal texture of the tongue; generated using morphological dilation operation. It is used to cover and preserve the concave morphological features of the tongue edge and the surrounding background.

[0014] Fourth, edge enhancement and feature concatenation steps. The Scharr edge detection operator is introduced to perform convolution calculations on the shallow feature S1 to obtain the gradient magnitude map. Using inner masks for mid-layer features Purification to generate internal features Meanwhile, edge features are obtained by filtering the gradient map using an outer mask. .

[0015] in The calculation method is as shown in equation (1). The calculation method is as shown in equation (2):

[0016]

[0017]

[0018] Subsequently, a visual fusion feature combining high-frequency edges and deep semantics was generated through channel splicing. The calculation method is as shown in equation (3):

[0019]

[0020] In equations (1) to (3) This indicates element-wise multiplication. Indicates downsampling, This indicates a concatenation operation along the channel dimension. Indicates the inner mask. Indicates the outer mask. This represents the obtained internal feature map. This represents the obtained edge feature map.

[0021] Fifth, the pulse-guided feature modulation step. A feature linear modulation (FiLM) mechanism is employed, using numerical pulse feature vectors to generate affine transformation parameters. These parameters are then used to perform channel-level scaling and offset transformations on the visual fusion features, achieving dynamic correction of visual features by non-visual priors. The calculation method is shown in equation (4):

[0022]

[0023] In formula (4) Indicates the scaling factor. Indicates the offset coefficient. Input visual features.

[0024] Sixth, the result output step. The multimodal fusion features modulated by the pulse image are input into the pre-constructed classification and decoding network, and the tongue segmentation mask and tongue image feature classification results are output.

[0025] Furthermore, this invention also provides a training method for a dual-masked multimodal tongue image analysis model that integrates pulse priors. This method is used to train the model used in the above analysis method and mainly includes the following steps:

[0026] The first stage of training involves freezing the parameters of the classification branch and the pulse guidance module, and optimizing the backbone network and segmentation decoder using only the segmentation loss function.

[0027] The second stage of training involves unfreezing the entire network parameters and introducing a deep supervision mechanism. Pure visual features are extracted before pulse feature fusion and input into the auxiliary classification head, outputting auxiliary prediction results. A total loss function is constructed, comprising segmentation loss, main classification loss based on multimodal fusion features, and auxiliary classification loss based on pure visual features. The calculation method is as shown in equation (5):

[0028] In equation (5), To divide the loss, The main classification loss is based on multimodal fusion features. For auxiliary classification loss based on purely visual features, The balance coefficient is denoted as , where and The calculation method is as shown in equation (6):

[0029]

[0030] In equation (6), This represents the total number of classification tasks. For the first The true label of each task The predicted probability of the model. In order to target the Dynamic weighting coefficients set for task classes.

[0031] Furthermore, to improve the convergence performance and global search capability of the multi-task model, a cosine annealing learning rate adjustment strategy with hot restart is introduced during model training. The calculation method is as shown in equation (7):

[0032]

[0033] In equation (7) and These represent the initial maximum and minimum values ​​of the learning rate, respectively. For the first The length of the restart cycle. This represents the number of rounds executed within the current cycle.

[0034] Finally, the trained model is deployed on the terminal. After inputting the data to be detected, it outputs the tongue segmentation mask and the multi-label classification results of the tongue image.

[0035] Beneficial effects

[0036] Compared with existing technologies, this invention, through its unique architecture design and training strategy, produces the following beneficial effects:

[0037] Effect 1: Resolved the spatial conflict between edge morphological features and internal texture extraction.

[0038] This invention constructs a dual-mask feature decoupling mechanism, using an inner mask to shield the oral cavity background to purify internal texture features, and an outer mask to preserve the tongue edge region. This "divide and conquer" strategy avoids the problem of traditional single edge segmentation masks accidentally removing edge depression features such as "teeth marks" when filtering out the background, thus ensuring the integrity of feature extraction.

[0039] Effect 2: Improved detection sensitivity for subtle high-frequency features.

[0040] To address the issue of texture loss caused by downsampling in deep neural networks, this invention introduces an edge enhancement branch based on the Scharr operator. The Scharr operator is isotropic to rotational changes, enabling it to capture subtle cracks and edge gradients in different growth directions. Furthermore, by using skip connections, it directly injects high-frequency details into the fused features, enhancing the model's ability to recognize early, minute morphological changes.

[0041] Effect 3: Effective correction of visual features by non-visual priors is achieved.

[0042] This invention employs the FiLM feature linear modulation mechanism to transform discrete pulse labels into continuous channel attention weights. This mechanism can dynamically enhance the response values ​​of visual features (such as greasy texture) that are strongly correlated with specific pulse types (such as slippery pulses) along the feature channel dimension, thus solving the problem of insufficient feature discrimination power of a single visual modality when facing complex sample categories from an algorithmic perspective.

[0043] Effect 4: Improved training convergence of multi-task models. Through a two-stage deep supervised learning strategy, the feature extractor is first solidified for segmentation task warm-up, and then auxiliary loss is introduced for joint training. This strategy suppresses model divergence caused by multi-task gradient interference and ensures simultaneous improvement in segmentation accuracy and classification accuracy. Attached Figure Description

[0044] Figure 1 This is a schematic diagram of the overall process of a dual-mask multimodal tongue image analysis method that integrates pulse image priors, provided in an embodiment of the present invention.

[0045] Figure 2 This is a schematic diagram of the structure of the dual-stream multimodal feature extraction backbone network constructed in this embodiment of the invention.

[0046] Figure 3 This is a schematic diagram illustrating the principle of the dual-mask feature decoupling and Scharr edge enhancement module in this embodiment of the invention.

[0047] Figure 4This is a schematic diagram of the structure of the pulse guidance module and the FiLM characteristic linear modulation mechanism in an embodiment of the present invention.

[0048] Figure 5 This is an example of the visualization results of the embodiment of the present invention on the test set; wherein, a) is the original input image, b) is the pulse feature label, c) is the tongue body segmentation ground truth, d) is the segmentation mask output by the method, e) is the Scharr edge enhancement feature map, f) is the classification ground truth label, and g) is the classification prediction result output by the method. Detailed Implementation

[0049] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0050] It should be noted that, in the subsequent description of this embodiment, the mathematical expressions of the formulas and the technical meanings of the symbols are consistent with the descriptions in the aforementioned "Summary of the Invention" section unless otherwise specified. Furthermore, the following embodiments are only used to explain the present invention and are not intended to limit the scope of protection of the present invention.

[0051] Furthermore, it should be specifically noted that the terms "spleen deficiency," "phlegm-dampness," "qi stagnation," and "excessive heat" used in this specification and accompanying drawings refer only to image classification labels defined based on professionally labeled data in this application. These labels are used to characterize specific combinations of texture, color, and morphology in tongue images. The classification results output by the method described in this application are only used as analytical data for image features and are not a final medical diagnostic conclusion regarding the subject's physiological health status.

[0052] like Figure 1 As shown in the figure, this embodiment proposes a dual-mask multimodal tongue image analysis method that integrates pulse priors. The specific processing flow is as follows:

[0053] Example Step 1: Acquisition and Preprocessing of Multimodal Data

[0054] This embodiment first uses a portable image acquisition terminal to acquire an RGB image of the tongue of the object to be detected, and then performs preprocessing:

[0055] Image standardization: The acquired original image is cropped by region of interest (ROI) and the resolution is uniformly adjusted to 768×576 pixels, and the pixel values ​​are normalized to the [0,1] range.

[0056] Pulse feature vectorization: Obtain the pulse description text of the object to be detected, and extract key terms describing the "position, number, shape, momentum, and force" attributes of the pulse. A 5-dimensional orthogonal feature space for pulse is constructed, with the five dimensions defined as follows: (Weak / Subtle) (Phlegm-dampness / slippery texture) (Qi stagnation / string tension) (Actual heat / flood number) (Inner cold / slow).

[0057] The extracted terms are mapped to 5-dimensional numerical vectors. For example, if the medical record is described as "slippery and rapid pulse", the corresponding vector is [0,1,0,1,0]; if it is described as "deep, thin and weak pulse", the corresponding vector is [1,0,0,0,0].

[0058] Example Step 2: Constructing a Two-Stream Feature Extraction Backbone Network

[0059] like Figure 2 As shown, a parallel two-stream network is constructed:

[0060] Step 2-1: Employ a ConvNeXt-V2-Base structure as the backbone to perform hierarchical feature extraction on the input image. The input image undergoes successive convolution and downsampling operations, outputting three layers of feature maps:

[0061] Shallow feature S1: downsampling ratio of 4x, resolution of 192×144, and number of channels of 128. This layer preserves rich spatial texture details.

[0062] Mid-layer feature S2: downsampling ratio of 16x, resolution of 48×36, and number of channels of 512. This layer serves as the core layer for feature fusion.

[0063] Deep feature S3: downsampling ratio of 32x, resolution of 24×18, and number of channels of 1024. This layer is mainly used to generate semantic segmentation masks.

[0064] Step 2-2: Introduce the Swing Transformer branch to compensate for the limited receptive field of convolution operations. This includes the following two core steps:

[0065] Feature injection: The mid-level features S2 output from the CNN branch are adjusted in number of channels through 1×1 convolution and flattened into serialized tokens, which are then used as the input embeddings of the Transformer.

[0066] Global modeling: Utilizes the multi-head self-attention (MSA) mechanism to calculate global feature associations and outputs a global semantic feature vector.

[0067] Example Step 3: Dual Mask Feature Decoupling

[0068] To resolve the conflict between extracting edge tooth marks and internal tongue features, this embodiment designs the following... Figure 3 The double mask generation module shown:

[0069] Step 3-1, Probability Map Generation: Upsample and decode the deep feature S3 using a segmentation decoder to output a tongue segmentation probability map with the same size as the original image. , Represents pixels The probability that it belongs to the tongue.

[0070] Step 3-2 Figure 3 The inner mask generation module shown in the left branch:

[0071] Set threshold Probability map of tongue segmentation Binarization. Then, a morphological erosion operation is performed on the binary image using a 3×3 structuring element to generate an inner mask. Because the resolution of the inner mask is different from the original... Figure 1 Since the resolution of the middle layer feature map S2 is 1 / 16 of the original image, the nearest neighbor interpolation algorithm is used to mask the inner layer. Downsampled to the same spatial dimension as S2, the internal feature map is calculated by calling the aforementioned equation (1). :

[0072]

[0073] Step 3-3 Figure 3 The outer mask generation module shown in the right branch:

[0074] A morphological dilation operation with a kernel size of 15×15 is performed on the segmentation probability map to generate an outer mask. This mask extends the region of interest outward, ensuring coverage of the tongue's edge recesses and the surrounding background, specifically designed for tooth mark detection.

[0075] Example Step 4: Edge Enhancement and Feature Concatenation

[0076] like Figure 3 As shown in the outer mask branch on the right and the bottom feature fusion module, this embodiment constructs an edge enhancement bypass. The core of this step is to introduce an explicit edge gradient flow.

[0077] Step 4-1: Edge Feature Extraction

[0078] Gradient magnitude map of shallow feature S1 extracted using the Scharr operator The gradient map is filtered using an outer mask, and the edge features are obtained by calling the aforementioned equation (2). :

[0079]

[0080] Step 4-2: Visual Feature Fusion

[0081] The pure internal features and edge features are concatenated along the channel dimension, and the visual fusion features are calculated by calling the aforementioned equation (3). :

[0082]

[0083] In equation (3), This indicates a channel-level concatenation operation. This operation does not change the spatial resolution of the feature map; instead, it concatenates the two items within the parentheses, representing the clean internal features. Edge enhancement features aligned with downsampling Stacking is performed along the channel dimension. The resulting visual fusion features are then stitched together. The total number of channels equals and The sum of the number of channels allows for the simultaneous preservation of internal texture semantic information and high-frequency geometric information at the edges within the same spatial coordinates.

[0084] Example Step 5: Pulse-guided FiLM feature modulation

[0085] like Figure 4 As shown, the core of this step lies in dynamically reweighting the visual channel using pulse vectors. Specifically, the system first inputs the preprocessed 5D numerical pulse vector into a multilayer perceptron (MLP) containing an input layer, hidden layer, and output layer for nonlinear encoding. The pulse vector... Mapped to scaling factor and offset coefficient Multimodal fusion features The calculation method is as described in equation (4) above:

[0086]

[0087] Through this channel-level affine transformation, the model can be based on pulse priors (such as the correspondence between slippery pulse and other pulse patterns). (Increase), adaptively enhance the response value of specific visual texture channels.

[0088] Example Step 6: Two-Stage Deep Supervision Training Strategy

[0089] This embodiment employs a phased optimization strategy to train the model, and the specific process is as follows:

[0090] Step 6-1: Task Warm-up:

[0091] First, the weight parameters of the classification head, pulse guidance module, and edge enhancement branch are set to an untrainable state. Only the segmentation loss is used. Gradient updates are performed on the backbone network and the segment decoder. This phase ends when the set number of pre-training epochs (e.g., Epoch=50) is reached or when the Dice coefficients on the segment validation set no longer increase.

[0092] Step 6-2: Joint optimization of all parameters

[0093] Unfreeze the parameters of all layers in the entire network and activate the auxiliary classification branch, which is directly derived from the pure visual feature layer before pulse fusion. Construct the total loss function. Used for end-to-end training, where The definition is as described in equation (5) above:

[0094]

[0095] in and The calculation method is as described in equation (6) above:

[0096]

[0097] Segmentation loss in equation (5) An equal-weighted combination of Dice Loss and BCE Loss is used to balance the problem of uneven sample region sizes; classification loss and All use the multi-label weighted cross-entropy shown in Equation (6) above; deep supervision weight coefficients The value was set to 0.4 to balance the supervision intensity between the multimodal fusion head and the pure vision-assisted head.

[0098] In equation (6), This represents the total number of classification tasks. In this example, it corresponds to 8 tongue appearance indicators. The dynamic weight coefficients are dynamically initialized based on the reciprocal of the frequency of each tongue appearance feature in the training set samples.

[0099] Step 6-3: Cosine annealing hot restart optimization

[0100] At the optimizer level, a cosine annealing strategy with hot restart is employed. Learning rate The calculation method is as shown in equation (7):

[0101]

[0102] In this embodiment, the values ​​of each parameter in equation (7) are set as follows:

[0103] Initial maximum learning rate Set as Minimum learning rate Set as Hot restart cycle Set to 50 epochs. When At that time, the learning rate will be reset to By leveraging the large step size of parameter updates brought about by the high learning rate after restarting, the model can escape the low-precision local minima and explore better solutions on the loss surface.

[0104] Through the above-described embodiments, the present invention can significantly improve the ability to identify subtle abnormal morphological features and intrinsic pathological attributes while ensuring segmentation accuracy.

[0105] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A dual-mask multimodal tongue image analysis method integrating pulse priors, characterized in that, Includes the following steps: Step 1: Obtain the tongue image of the object to be detected, and the corresponding numerical pulse feature vector; Step 2: Construct a multimodal fusion backbone network, and use the convolutional neural network branch to extract the shallow features (S1) and mid-level features (S2) of the tongue image. In parallel, use the Transformer branch to extract global semantic features based on the feature injection mechanism. Step 3: Construct a dual-mask feature decoupling module, and use the segmentation probability map output by the network to generate an inner mask covering the internal region of the tongue and an outer mask covering the edge of the tongue and the surrounding background. Step 4: Construct an edge enhancement and feature fusion module. Use the Scharr operator to extract the gradient magnitude of the shallow feature (S1), combine it with the outer mask to generate edge enhancement features, and then perform channel splicing with the middle feature (S2) after filtering by the inner mask to generate visual fusion features. Step 5: Construct a pulse-guided feature modulation module, generate affine transformation parameters using the numerical pulse feature vector, and perform channel-level dynamic correction on the visual fusion features through Feature Linear Modulation (FiLM) mechanism; Step 6: Output the results: Input the multimodal fusion features into the pre-constructed classification and decoding network, and output the tongue segmentation mask and tongue image feature classification results.

2. The method for dual-masked multimodal tongue image analysis based on pulse prior as described in claim 1, characterized in that, Step 1 includes the following steps: Step 1-1: Using the image acquisition module of a portable smart terminal, acquire the original image of the tongue of the object to be detected under natural lighting conditions; Step 1-2: Perform region of interest cropping and resolution normalization processing on the original image of the tongue to generate a fixed-size image of the tongue to be detected; Step 1-3: Obtain the pulse description text of the object to be detected and extract keywords describing the pulse morphology and properties. Steps 1-4: Construct a 5-dimensional orthogonal feature space for pulse diagnosis, including position, number, shape, potential, and force; Steps 1-5: Map the keywords to the feature space to generate a 5-dimensional numerical pulse feature vector containing the dimensions of "deficiency and thinness, phlegm and dampness, qi stagnation, excess heat, and internal cold".

3. The method for dual-masked multimodal tongue image analysis based on fusion pulse prior as described in claim 1, characterized in that, Step 2 specifically includes: the convolutional neural network branch adopts a hierarchical downsampling structure, outputting shallow features (S1) downsampled by 4 times, mid-level features (S2) downsampled by 16 times, and mid-level features (S3) downsampled by 32 times; the feature injection mechanism specifically involves: adjusting the number of channels of the mid-level features (S2) through the convolutional layer and flattening them into serialized words, which are then used as input embeddings injected into the Transformer branch, and global context modeling is performed using a multi-head self-attention mechanism.

4. The method for dual-masked multimodal tongue image analysis based on pulse priors as described in claim 1, characterized in that, Step 3 specifically includes: Inner mask generation: Binarizing the segmentation probability map and performing morphological erosion using structuring elements to generate an inner mask, which is used to shield the oral cavity background outside the tongue contour during feature extraction. Outer mask generation: Performing morphological dilation on the segmentation probability map to generate an outer mask, the outer mask covering an area extending outward from the tongue body to preserve the concave features of the tongue edge.

5. The dual-mask multimodal tongue image analysis method based on pulse prior as described in claim 1, characterized in that, Step 4 specifically includes: performing Scharr convolution operations on the shallow feature (S1) using horizontal and vertical convolution kernels respectively to calculate the gradient magnitude map; multiplying the gradient magnitude map element-wise with the outer mask to filter out background noise and obtain the edge enhancement feature; multiplying the inner mask element-wise with the middle feature (S2) to obtain the clean internal feature, and concatenating it with the edge enhancement feature in the channel dimension to generate the visual fusion feature.

6. The method for dual-masked multimodal tongue image analysis based on pulse priors as described in claim 1, characterized in that, Step 5 includes the following steps: Step 5-1, using a multilayer perceptron to perform nonlinear encoding on the numerical pulse feature vector and output scaling coefficients. With offset coefficient Step 5-2: According to the formula Visual fusion features Perform channel transformation to obtain multimodal fusion features. The transformation is used to enhance or suppress the response of specific visual feature channels based on pulse priors.

7. A dual-masked multimodal tongue image analysis system integrating pulse priors, characterized in that, include: The data acquisition module is used to acquire tongue images and numerical pulse feature vectors; The feature extraction module is used to extract multi-scale visual features through a dual-stream backbone network; The dual-mask generation module is used to generate inner and outer masks based on the segmentation probability map. The edge enhancement module is used to extract edge enhancement features using the Scharr operator and the outer mask; A multimodal fusion module is used to modulate visual features using pulse feature vectors via the FiLM mechanism; The output module is used to output the tongue segmentation results and tongue image feature classification results; The system is used to perform the method as described in any one of claims 1 to 6.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 6.

Images