Interactive voiceprint clustering method, system, electronic device and storage medium

Abstract

The present invention relates to the field of voiceprint clustering technology, and in particular to an interactive voiceprint clustering method, system, electronic device and storage medium. The method comprises: extracting feature vectors of voice files, and clustering based on a preset clustering threshold; reviewing each type of voice files after clustering to confirm the final number of people contained in the voice files; re-clustering according to the confirmed number of people, and calculating the mean value based on the voiceprint features of each type as the central feature of the voiceprint features of that type, obtaining the similarity between each feature in the type and the central feature, and based on a preset similarity threshold, marking the files below the similarity threshold as difficult-to-classify files; reviewing the difficult-to-classify files again, and reallocating them to the correct category. The present invention combines the advantages of algorithms and humans in an interactive way through the combination of humans and algorithms, solves the deficiencies in the algorithms, and thus improves the effect of the final clustering.

Description

Technical Field

[0001] The present invention relates to the technical field of voiceprint clustering, and in particular to an interactive voiceprint clustering method, system, electronic device and storage medium. Background Art

[0002] Voiceprint clustering is to group the voice files of the same person into one category. The existing technology uses algorithms to extract voiceprint features in speech, and classifies the ones with high similarity into one category by feature comparison. The clustering method generally sets a similarity threshold or sets the number of people as the clustering termination condition.

[0003] The accuracy of clustering is limited by the algorithm itself. Using the number of people as the termination condition will have a better clustering effect, but the algorithm cannot accurately obtain the number of people information. Using the similarity threshold as the termination condition has a relatively poor effect. Summary of the invention

[0004] In view of the problem that the voiceprint clustering method in the prior art that sets a similarity threshold or sets the number of people as the clustering termination condition is limited in accuracy and cannot accurately obtain the number of people information, the present invention provides an interactive voiceprint clustering method, system, electronic device and storage medium, which utilizes interaction with people to confirm the number of people in the classification and deal with the problem of difficult-to-classify speech.

[0005] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0006] In a first aspect, in an embodiment provided by the present invention, an interactive voiceprint clustering method is provided, comprising the following steps:

[0007] Extract feature vectors of voice files and cluster them based on a preset clustering threshold;

[0008] Reviewing each type of voice file after clustering to confirm the final number of people included in the voice file;

[0009] Re-clustering based on the number of confirmed people, and calculating the mean value based on the voiceprint features of each category as the central feature of the voiceprint features of that category, calculating the similarity between each feature in the category and the central feature, and based on a preset similarity threshold, marking those below the similarity threshold as difficult-to-classify files;

[0010] The difficult-to-classify documents are reviewed again and reallocated to the correct category.

[0011] In some embodiments provided by the present invention, the feature vector of the voice file is extracted by a voiceprint extraction algorithm, and the method for extracting the feature vector of the voice file comprises the following steps:

[0012] Get the voice file to be extracted;

[0013] Performing voice recognition on the acquired voice file to obtain audio features of the voice file;

[0014] The obtained audio features are input into the model to obtain the feature vector of the voice file.

[0015] In some embodiments provided by the present invention, voiceprint feature extraction is performed on the voice file, and a feature vector of the voice file is extracted using an ivector or xvector algorithm to obtain the feature vector of the voice file.

[0016] In some embodiments provided by the present invention, when clustering is performed based on a preset clustering threshold, similarity is calculated for different feature vectors extracted from the voice file, wherein the similarity of different feature vectors is calculated according to a cosine or PLDA similarity algorithm.

[0017] In some embodiments provided by the present invention, when clustering is performed based on a preset clustering threshold, the AHC clustering method or the SC clustering method is used for clustering.

[0018] In some embodiments provided by the present invention, the preset clustering threshold is a threshold result confirmed by testing. Before confirming the threshold result, a clustering test is performed through a given test file, and the threshold with the best effect is selected as the default threshold used later.

[0019] In some embodiments provided by the present invention, each type of voice file after clustering is audited manually, and judgment is made by listening to the voice of the speaker in the recording file, and different types but belonging to the same person are merged into the same type.

[0020] In some embodiments provided by the present invention, when the difficult-to-classify documents are reviewed again, manual review is performed to make up for the deficiencies of the algorithm and reallocate them to the correct category.

[0021] In a second aspect, in another embodiment provided by the present invention, an interactive voiceprint clustering system is provided, comprising:

[0022] A feature extraction unit, used to perform speech recognition on the acquired speech file to extract feature vectors;

[0023] A clustering unit, used for clustering the extracted feature vectors based on a preset clustering threshold;

[0024] a similarity comparison unit, configured to use the mean value of the voiceprint features of each class after the number of people is re-clustered as the central feature of the voiceprint features of that class, calculate the similarity between each feature in the class and the central feature, and compare based on a preset similarity threshold, marking files with a similarity below the threshold as difficult to classify files; and

[0025] The review unit is used to review each type of voice file after clustering to confirm the final number of people included in the voice file, and is also used to review the difficult-to-classify files again and reallocate them to the correct category.

[0026] In a third aspect, in another embodiment provided by the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of an interactive voiceprint clustering method when loading and executing the computer program.

[0027] In a fourth aspect, in yet another embodiment provided by the present invention, a storage medium is provided, storing a computer program, and when the computer program is loaded and executed by a processor, the steps of the interactive voiceprint clustering method are implemented.

[0028] The technical solution provided by the present invention has the following beneficial effects:

[0029] The interactive voiceprint clustering method, system, electronic device and storage medium of the present invention extracts the feature vector of the voice file, performs manual review after clustering, merges different classes in each class of voice files after clustering but belonging to the same person into the same class, confirms the final number of people according to the final number of merged classes, and uses an algorithm to mark difficult-to-classify files according to similarity, and performs manual review to reallocate difficult-to-classify files to the correct class. Through the combination of humans and algorithms, the advantages of algorithms and humans are combined in an interactive way, the deficiencies of the algorithms are solved, and the final clustering effect is improved.

[0030] These and other aspects of the present invention will be more clearly understood in the following description of the embodiments. It should be understood that the above general description and the following detailed description are only exemplary and explanatory and cannot limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0032] Figure 1 The present invention is a flowchart of an interactive voiceprint clustering method.

[0033] Figure 2 The figure is a flow chart of feature vector extraction in an interactive voiceprint clustering method in one embodiment of the present invention.

[0034] Figure 3 The system block diagram of an interactive voiceprint clustering system in one embodiment of the present invention.

[0035] Figure 4FIG. 1 is a structural block diagram of an electronic device in one embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0037] In some of the processes described in the specification and claims of the present invention and the above-mentioned drawings, multiple operations appearing in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel.

[0038] The following will be combined with the drawings in the exemplary embodiments of the present invention to clearly and completely describe the technical solutions in the exemplary embodiments of the present invention. Obviously, the exemplary embodiments described are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0039] Figure 1 A flowchart of an interactive voiceprint clustering method provided by the present invention.

[0040] See also Figure 1 As shown, the present invention provides an interactive voiceprint clustering method, which utilizes interaction with people to confirm the number of people to be classified and handles the problem of difficult-to-classify speech.

[0041] In this embodiment, the interactive voiceprint clustering method includes the following steps:

[0042] S1, extracting feature vectors of voice files and clustering them based on a preset clustering threshold;

[0043] S2, reviewing each type of voice file after clustering to confirm the final number of people included in the voice file;

[0044] S3, re-clustering according to the number of confirmed people, and calculating the mean value based on the voiceprint features of each category as the central feature of the voiceprint features of this category, obtaining the similarity between each feature in the category and the central feature, and based on a preset similarity threshold, marking the files below the similarity threshold as difficult to classify files;

[0045] S4. Review the difficult-to-classify documents again and reallocate them to the correct category.

[0046] The interactive voiceprint clustering method of the present embodiment is divided into the above four main steps, wherein the clustering processing of voice files and the marking of difficult-to-classify files are voiceprint clustering operations completed by algorithms, and the review of each type of voice files after clustering and the re-review of the difficult-to-classify files are completed by manual review. The interactive voiceprint clustering method of the present invention combines the respective advantages of algorithms and humans in an interactive way through the combination of humans and algorithms, solves the shortcomings of the algorithms, and thus improves the final clustering effect.

[0047] In step S1 of this embodiment, the feature vector of the voice file is extracted by a voiceprint extraction algorithm, see Figure 2 As shown, the method for extracting the feature vector of the voice file comprises the following steps:

[0048] S101, obtaining a voice file to be extracted;

[0049] S102, performing voice recognition on the acquired voice file to obtain audio features of the voice file;

[0050] S103: Input the obtained audio features into the model to obtain a feature vector of the voice file.

[0051] In this embodiment, the algorithm for extracting voiceprints adopts the current mainstream ivacor, xvector, etc. The specific steps are other front-end features of audio files, such as mfcc or fbank, etc. In this embodiment, voiceprint features are extracted from voice files, and feature vectors of voice files are extracted using ivacor or xvector algorithms, which are then input into the model to finally obtain feature vectors.

[0052] In this embodiment, when performing clustering operation, different feature vectors are similarity calculated according to PLDA algorithm or Cosine algorithm, wherein PLDA algorithm or Cosine algorithm is an existing resident similarity calculation algorithm. In this embodiment, the algorithm for similarity during clustering includes but is not limited to PLDA algorithm or Cosine algorithm.

[0053] Therefore, when clustering is performed based on a preset clustering threshold, the similarity of different feature vectors extracted from the voice file is calculated, wherein the similarity of different feature vectors is calculated according to a cosine or PLDA similarity algorithm.

[0054] In this embodiment, when clustering is performed based on a preset clustering threshold, an AHC clustering method or a SC clustering method is used for clustering. It should be noted that when clustering is performed based on a preset clustering threshold, the clustering method includes but is not limited to the AHC clustering method or the SC clustering method.

[0055] In this embodiment, the preset clustering threshold is a threshold result confirmed by testing. Before confirming the threshold result, a clustering test is performed using a given test file, and the threshold with the best effect is selected as the default threshold used later.

[0056] When clustering is performed based on a preset clustering threshold, the preset clustering threshold will use a higher value so that each class contains the same person. After the above clustering, the number of clusters is usually greater than the actual number of clustered people, but the number of clusters after clustering is much smaller than the number of voice files. The clustering operation can save a lot of time for manual clustering, improve the efficiency of voiceprint clustering operations, and facilitate subsequent manual review.

[0057] For example: a voiceprint extraction algorithm in one embodiment of the present invention has a preset clustering threshold of 0.7 and a higher threshold of 0.8. There are 20 voice files clustered and divided into five categories. Since a higher threshold is used for clustering, each category basically contains the voice of the same person. Then, the human only needs to listen to any one of the voices in each category. There are 5 categories here, so only 5 voices need to be listened to, and then the same categories are merged. Here, it is assumed that two categories are manually merged, and finally the 20 files are divided into 4 categories. In this embodiment, different algorithms calculate similarity in different ways, so the higher threshold is also relative.

[0058] In step S2 of this embodiment, each type of voice file after clustering is manually audited by listening to the voice of the speaker in the recording file to make a judgment, and different types but belonging to the same person are merged into the same type.

[0059] During manual review, voice files of different categories but belonging to the same person are merged into the same category based on listening to each category of voice files, and the final number of people is confirmed based on the number of categories finally merged. Since the threshold in the previous step is high, the same person will be divided into two people. In this step, the number of people is further merged manually to confirm the number of people. The higher threshold in the previous step is used to confirm the number of people. As a result, there will be more classifications than the normal threshold, and each category will be purer, which is better combined with manual operation.

[0060] In this embodiment, manual review is performed by listening to the voice of the speaker in the recording file. There will be many clustered files here, but the actual number of speakers will not be many, so manual judgment is feasible. For example, two voice files may have low similarity scores due to noise, differences in collection equipment or defects in the algorithm itself, but this problem can be compensated by humans, and it is possible to judge whether the two voices are from the same person.

[0061] Re-cluster according to the number of people confirmed in step S2, and calculate the mean value based on the voiceprint features of each class as the central feature of the class. In this embodiment, the voiceprint feature is a one-dimensional vector that can characterize the voiceprint calculated by the algorithm. When calculating the mean value as the central feature of the class, for example, there are three files in total, and the extracted voiceprint vectors are A, B, and C. Then the central feature is 1 / 3*(A+B+C).

[0062] When calculating the similarity between each feature and the central feature in the class, the similarity has a related calculation method that is the same as step S2, and can use, for example, cosine or plda algorithm. Files with a similarity lower than a preset low threshold are difficult to classify, or are called files with low similarity.

[0063] In step S3, since the algorithm generally clusters based on the number of people better than clustering based on the similarity threshold, clustering is performed again after the number of people is confirmed to improve the accuracy of clustering. At the same time, this step uses forced classification based on the number of people, and some people with low similarity will be classified into one category. This category may be misclassified due to algorithm problems and needs further confirmation.

[0064] Therefore, when the difficult-to-classify voice files are manually reviewed again, step S3 marks the voices with low similarity as difficult to classify, and in step S4, the deficiencies of the algorithm are compensated manually, thereby further improving the accuracy of classification.

[0065] In this embodiment, when the difficult-to-classify files are reviewed again, manual review is performed to make up for the deficiencies of the algorithm and reallocate them to the correct category.

[0066] It should be understood that, although described in a certain order, these steps are not necessarily performed in sequence in the above order. Unless there is clear explanation in this article, the execution of these steps does not have strict order restriction, and these steps can be performed in other orders. Moreover, a part of the steps of the present embodiment may include a plurality of steps or a plurality of stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a part of the steps or stages in other steps or other steps.

[0067] In one embodiment, see Figure 3 As shown, an embodiment of the present invention provides an interactive voiceprint clustering system, which includes a feature extraction unit 100, a clustering unit 200, a similarity comparison unit 300 and an audit unit 400.

[0068] The feature extraction unit 100 is used to extract feature vectors by performing voice recognition on the acquired voice file. In this embodiment, the algorithm for extracting voiceprints adopts the currently mainstream ivacor, xvector, etc. In this embodiment, the voice file is subjected to voiceprint feature extraction, and the feature vector of the voice file is extracted by using ivacor or xvector algorithm, which is then input into the model to finally obtain the feature vector.

[0069] The clustering unit 200 is used to cluster the extracted feature vectors based on a preset clustering threshold. When clustering based on the preset clustering threshold, the clustering unit 200 uses AHC clustering or SC clustering method for clustering.

[0070] It should be noted that when clustering is performed based on a preset clustering threshold, the clustering method includes but is not limited to the AHC clustering method or the SC clustering method.

[0071] The similarity comparison unit 300 is used to take the mean value of the voiceprint features of each category after re-clustering the number of people as the central feature of the voiceprint features of that category, calculate the similarity between each feature in the category and the central feature, and compare based on a preset similarity threshold, and mark those below the similarity threshold as difficult-to-classify files.

[0072] When the similarity comparison unit 300 performs clustering operation, different feature vectors perform similarity calculation according to the PLDA algorithm or the Cosine algorithm, wherein the PLDA algorithm or the Cosine algorithm is an existing resident similarity calculation algorithm. In this embodiment, the algorithm for similarity during clustering includes but is not limited to the PLDA algorithm or the Cosine algorithm.

[0073] When clustering is performed based on a preset clustering threshold, the similarity of different feature vectors extracted from the voice file is calculated, wherein the similarity of different feature vectors is calculated according to the similarity algorithm of cosine or plda.

[0074] The review unit 400 is used to review each type of voice file after clustering to confirm the final number of people included in the voice file, and is also used to review the difficult-to-classify files again and reallocate them to the correct category.

[0075] The review unit 400 is usually reviewed manually. During the manual review, different categories but belonging to the same person are merged into the same category based on listening to the voice files of each category, and the final number of people is confirmed based on the final number of merged categories. Since the threshold of the previous step is high, the same person will be divided into two people. This step is done manually to further merge and confirm the number of people. The higher threshold of the previous step is used to confirm the number of people. As a result, there will be more classifications than the normal threshold, and each category obtained will be purer, which is better combined with manual operation.

[0076] In this embodiment, the interactive voiceprint clustering system adopts the steps of the aforementioned interactive voiceprint clustering method during execution. Therefore, the operation process of the interactive voiceprint clustering system will not be described in detail in this embodiment.

[0077] In one embodiment, Figure 4 FIG. 1 is a block diagram of an electronic device according to an embodiment of the present invention. In a possible design, Figure 4 As shown, in an embodiment of the present invention, an electronic device 600 is provided, the electronic device 600 includes a memory 601 and a processor 602, the memory 601 stores a computer program, and the processor 602 is configured to execute the computer program stored in the memory 601. The memory 601 is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor 602 to implement the steps in the above method embodiment:

[0078] Extract feature vectors of voice files and cluster them based on a preset clustering threshold;

[0079] Reviewing each type of voice file after clustering to confirm the final number of people included in the voice file;

[0080] Re-clustering based on the number of confirmed people, and calculating the mean value based on the voiceprint features of each category as the central feature of the voiceprint features of that category, calculating the similarity between each feature in the category and the central feature, and based on a preset similarity threshold, marking those below the similarity threshold as difficult-to-classify files;

[0081] The difficult-to-classify documents are reviewed again and reallocated to the correct category.

[0082] It should be noted that, according to an embodiment of the present invention, the method described above with reference to the accompanying drawings can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program tangibly contained on a readable medium, and the computer program includes program code for executing the method in the accompanying drawings. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part, and / or installed from a removable medium.

[0083] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the road map or block diagram can represent a unit, a program segment or a part of a code, and the unit, a program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0084] In an embodiment of the present invention, a storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented:

[0085] Extract feature vectors of voice files and cluster them based on a preset clustering threshold;

[0086] Reviewing each type of voice file after clustering to confirm the final number of people included in the voice file;

[0087] Re-clustering based on the number of confirmed people, and calculating the mean value based on the voiceprint features of each category as the central feature of the voiceprint features of that category, calculating the similarity between each feature in the category and the central feature, and based on a preset similarity threshold, marking those below the similarity threshold as difficult-to-classify files;

[0088] The difficult-to-classify documents are reviewed again and reallocated to the correct category.

[0089] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program represented by computer instructions, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory.

[0090] Non-volatile memory may include read-only memory, magnetic tape, floppy disk, flash memory or optical storage, etc. Volatile memory may include random access memory or external cache memory. As an illustration and not limitation, RAM may be in various forms, such as static random access memory or dynamic random access memory, etc.

[0091] In summary, the interactive voiceprint clustering method, system, electronic device and storage medium of the present invention extracts the feature vector of the voice file, performs manual review after clustering, merges different classes in each class of voice files after clustering but belonging to the same person into the same class, confirms the final number of people according to the final number of merged classes, and uses an algorithm to mark difficult-to-classify files according to similarity, and then performs manual review to reallocate difficult-to-classify files to the correct class. Through the combination of people and algorithms, the advantages of algorithms and people are combined in an interactive way to solve the shortcomings of the algorithm, thereby improving the final clustering effect.

[0092] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An interactive voiceprint clustering method, characterized in that: include: Extract feature vectors of voice files and cluster them based on a preset clustering threshold; Auditing each type of clustered voice files to confirm the final number of people included in the voice files, wherein the auditing of each type of clustered voice files is performed manually, and judgment is made by listening to the voice of the speaker in the recording file, and different types but the same person are merged into the same type; Re-clustering based on the number of confirmed people, and calculating the mean value based on the voiceprint features of each category as the central feature of the voiceprint features of that category, calculating the similarity between each feature in the category and the central feature, and based on a preset similarity threshold, marking those below the similarity threshold as difficult-to-classify files; The difficult-to-classify documents are reviewed again and reallocated to the correct category.

2. The interactive voiceprint clustering method according to claim 1, characterized in that: The feature vector of the voice file is extracted by a voiceprint extraction algorithm, and the method for extracting the feature vector of the voice file comprises the following steps: Get the voice file to be extracted; Performing voice recognition on the acquired voice file to obtain audio features of the voice file; The obtained audio features are input into the model to obtain the feature vector of the voice file.

3. The interactive voiceprint clustering method according to claim 2, characterized in that: The voice file is subjected to voiceprint feature extraction, and an ivector or xvector algorithm is used to extract a feature vector of the voice file to obtain the feature vector of the voice file.

4. The interactive voiceprint clustering method according to claim 1, characterized in that: When clustering is performed based on a preset clustering threshold, the similarity of different feature vectors extracted from the voice file is calculated, wherein the similarity of different feature vectors is calculated according to the similarity algorithm of cosine or plda.

5. The interactive voiceprint clustering method according to claim 1, characterized in that: When clustering is performed based on a preset clustering threshold, the AHC clustering method or the SC clustering method is used for clustering.

6. The interactive voiceprint clustering method according to claim 5, characterized in that: The preset clustering threshold is the threshold result confirmed by testing.

7. An interactive voiceprint clustering system, characterized by: The interactive voiceprint clustering system adopts the interactive voiceprint clustering method described in any one of claims 1 to 6 to improve the final clustering effect; The interactive voiceprint clustering system includes: A feature extraction unit, used to perform speech recognition on the acquired speech file to extract feature vectors; A clustering unit, used for clustering the extracted feature vectors based on a preset clustering threshold; a similarity comparison unit, configured to use the mean value of the voiceprint features of each class after the number of people is re-clustered as the central feature of the voiceprint features of that class, calculate the similarity between each feature in the class and the central feature, and compare based on a preset similarity threshold, marking files with a similarity below the threshold as difficult to classify files; and The review unit is used to review each type of voice file after clustering to confirm the final number of people included in the voice file, and is also used to review the difficult-to-classify files again and reallocate them to the correct category. The review of each type of voice file after clustering is performed manually, and judgment is made by listening to the voice of the speaker in the recording file, and different types but belonging to the same person are merged into the same type.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor loads and executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A storage medium storing a computer program, characterized in that: When the computer program is loaded and executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images