Speech recognition method, apparatus and device

Abstract

The application discloses a speech recognition method, device and equipment, and belongs to the multimedia technical field. The method comprises the following steps: acquiring audio data for target data; inputting the audio data into a speech recognition model to output a first text; correcting an entity word in the first text according to a first data set of the target data to obtain a second text, wherein the second text is a text translated from the audio data, and the first data set is a set of multiple entity words of the target data. It can be seen that the text output by the speech recognition model is corrected by using the data set of the target data. Since the target data can be data in a specific field, the data set of the target data can contain entity words in the specific field. Therefore, the application can effectively assist the speech recognition model in accurately recognizing the entity words in the specific field and improve the recognition accuracy of the entity words in the specific field.

Description

Technical Field

[0001] This application relates to the field of multimedia technology, and in particular to a speech recognition method, apparatus and device. Background Technology

[0002] Automatic speech recognition (ASR) is used to translate audio data into corresponding text. One related technology provides a speech recognition method that relies on an ASR model. The ASR model learns from a reference number of sample data and their corresponding sample texts to acquire the ability to translate audio data into text. Therefore, this speech recognition method typically performs audio processing on the audio data to be recognized, inputs the processed audio data into the ASR model, and outputs the corresponding text.

[0003] However, the audio processing of the audio data to be recognized may be affected by environmental noise, leading to lower accuracy. Furthermore, ASR models require a large amount of sample data for training, and the continuous updating of domain-specific entity words (such as entity words in emerging fields or custom entity words) is often limited. Therefore, the updated entity words have a relatively small impact on the training of the ASR model compared to the large amount of sample data, resulting in poor ability of the ASR model to recognize domain-specific entity words. Consequently, the accuracy of speech recognition methods in related technologies for recognizing domain-specific entity words is relatively low. Summary of the Invention

[0004] This application provides a speech recognition method, apparatus, and device that effectively improves the accuracy of recognizing entity words in a specific domain. The technical solution is as follows.

[0005] Firstly, a speech recognition method is provided, comprising: acquiring audio data for target material; inputting the audio data into a speech recognition model to output a first text; and correcting entity words in the first text based on a first dataset of the target material to obtain a second text, wherein the second text is the text translated from the audio data, and the first dataset is a set of multiple entity words of the target material. In this application, audio data to be recognized for a specified material is input into a speech recognition model to obtain a first text, and then a standard dataset (i.e., the dataset of the specified material) is used to correct the entity words in the first text to obtain the text translated from the audio data to be recognized. It is evident that this application corrects the text output by the speech recognition model using a dataset of the specified material. Since the dataset of the specified material can contain entity words from a specific domain, it can effectively assist the speech recognition model in accurately recognizing entity words from a specific domain, thereby improving the recognition accuracy of entity words from a specific domain.

[0006] The target data can include data from a specific domain. This means that the target data contains entity words from that specific domain. Therefore, the first dataset obtained based on the target data, and the text output by the speech recognition model corrected based on the first dataset, can effectively assist the speech recognition model in accurately recognizing entity words from that specific domain, thereby improving the accuracy of entity word recognition.

[0007] The first dataset of target data may include multiple entity words from data in a specific domain. Alternatively, the first dataset may include multiple entity words from the target data, or multiple entity words from the target data and their corresponding audio features.

[0008] In one possible implementation, the first dataset also includes audio features corresponding to multiple entity words. Based on the first dataset of target data, the entity words in the first text are corrected to obtain the second text. This includes: comparing the audio features corresponding to each entity word in the first dataset with the audio features of each entity word in the first text; determining the similarity between the first entity word and the second entity word if their audio features are the same; and correcting the second entity word in the first text to the first entity word if the similarity is greater than a threshold, thus obtaining the second text. Therefore, by comparing the audio features corresponding to each entity word in the first dataset with the audio features of each entity word in the first text, and if the similarity between the first entity word and the second entity word in the first text is greater than a threshold, the second entity word in the first text is corrected to the first entity word. This corrects the text output by the speech recognition model, assisting the speech recognition model in translating audio data and improving the accuracy of the translated text.

[0009] In one possible implementation, the method further includes: determining the first text as the second text if the similarity is less than or equal to a threshold. It can be seen that by comparing the audio features corresponding to each entity word in the first dataset with the audio features of each entity word in the first text, if the audio features corresponding to the first entity word in the first dataset and the second entity word in the first text are the same, and if the similarity between the first entity word and the second entity word is less than or equal to a threshold, the second entity word in the first text is not corrected. The accuracy of the entity words in the first text is determined by comparing the entity words in the first text and the first dataset, thereby improving the accuracy of the translated text from the speech data.

[0010] In one possible implementation, before correcting entity words in the first text based on the first dataset of the target material to obtain the second text, the method further includes: acquiring video data of the target material; inputting the video data into an optical character recognition (OCR) model to output a third text; and extracting entity words from the third text to obtain the first dataset. The optical character recognition model can be an OCR model. In this application, the first dataset is obtained through the video data of the target material. This uses the video data of the target material as reference data for correcting the output text of the speech recognition model, effectively improving recognition accuracy and reducing manual proofreading costs. Furthermore, the target material can contain information from a specific domain. This means that the target material contains entity words from a specific domain. Therefore, the first dataset obtained based on the target material, and the text output by the speech recognition model corrected based on the first dataset, can effectively assist the speech recognition model in accurately recognizing entity words from a specific domain, improving the recognition accuracy of entity words from that domain.

[0011] In one possible implementation, the video data includes multiple images. After acquiring the video data of the target material, the process further includes: filtering out multiple key images from the multiple images according to time; inputting the video data into an optical character recognition model and outputting third text, including: inputting multiple key images into the optical character recognition model and outputting third text. The video data can be a video stream composed of multiple images arranged in time. Therefore, the video data can include multiple images. Since the multiple images may include duplicate images and images without text information, it is necessary to filter out duplicate images and images without text information during the construction of the first dataset to reduce computational load.

[0012] In one possible implementation, multiple key images are selected from multiple images based on time, including: selecting multiple non-repeating images from multiple images as multiple key images based on time. In this way, by filtering the video data to select multiple non-repeating images as the data for creating the first dataset, unnecessary computation is reduced and power consumption is saved while ensuring the accuracy of the first dataset.

[0013] In one possible implementation, multiple key images are selected from multiple images based on time, including: selecting multiple images containing text information from multiple images as multiple key images based on time. In this way, by filtering video data to select multiple images containing text information as data for creating the first dataset, unnecessary computation is reduced and power consumption is saved while ensuring the accuracy of the first dataset.

[0014] In one possible implementation, multiple key images are selected from multiple images based on time. This includes: selecting multiple non-repeating images from the multiple images based on time, and then selecting multiple images containing text information from these multiple images based on time as multiple key images. In this way, by filtering video data to select valid data for the creation of the first dataset, unnecessary computation is reduced and power consumption is saved while ensuring the accuracy of the first dataset.

[0015] In one possible implementation, the entity words in the third text are extracted to obtain the first dataset, which includes: inputting the third text into an entity recognition model and outputting the first dataset. The entity recognition model can be a named entity recognition (NER) model.

[0016] Secondly, a speech recognition device is provided, comprising: an acquisition module for acquiring audio data of target material; an output module for inputting the audio data into a speech recognition model and outputting a first text; and a correction module for correcting entity words in the first text according to a first dataset of the target material to obtain a second text, wherein the second text is text translated from the audio data, and the first dataset is a set of multiple entity words of the target material.

[0017] In one possible implementation, the first dataset also includes audio features corresponding to multiple entity words; a correction module is used to: compare the audio features corresponding to each entity word in the first dataset with the audio features of each entity word in the first text; if the audio features corresponding to the first entity word in the first dataset and the second entity word in the first text are the same, determine the similarity between the first entity word and the second entity word; if the similarity is greater than a threshold, correct the second entity word in the first text to the first entity word to obtain the second text.

[0018] In one possible implementation, a correction module is used to determine that the first text is the second text if the similarity is less than or equal to a threshold.

[0019] In one possible implementation, an acquisition module is used to acquire video data of the target material; an input module is used to input the video data into an optical character recognition model and output third text; and an extraction module is used to extract each entity word from the third text to obtain a first dataset.

[0020] In one possible implementation, the video data includes multiple images; the device further includes: a filtering module for filtering multiple key images from the multiple images according to time; and an input module for inputting the multiple key images into an optical character recognition model and outputting third text.

[0021] In one possible implementation, a filtering module is used to filter out multiple non-repeating images from multiple images according to time as multiple key images.

[0022] In one possible implementation, a filtering module is used to filter multiple images containing text information from multiple images according to time as multiple key images.

[0023] In one possible implementation, an extraction module is used to input third text into the entity recognition model and output a first dataset.

[0024] Thirdly, an electronic device is provided, comprising a processor coupled to a memory; the memory stores at least one instruction, which is loaded and executed by the processor to enable the electronic device to implement the speech recognition method of claim 1.

[0025] Fourthly, a computer program (product) is provided, comprising: computer program code, which, when executed by a computer, causes the computer to perform the speech recognition methods described in the above aspects.

[0026] Fifthly, a computer-readable storage medium is provided that stores a program or instructions, wherein when the program or instructions are run on a computer, the speech recognition methods described in the above aspects are executed.

[0027] In a sixth aspect, a chip is provided, including a processor for retrieving and executing instructions stored in a memory, causing a computer equipped with the chip to perform the speech recognition methods described in the above aspects.

[0028] In a seventh aspect, another chip is provided, comprising: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected via an internal connection path. The processor is used to execute code in the memory. When the code is executed, the computer with the chip installed performs the speech recognition methods described in the above aspects.

[0029] It should be understood that the beneficial effects of the technical solutions and corresponding possible implementations of the third to seventh aspects of this application can be found in the above description of the technical effects of the first aspect and its corresponding possible implementations, and will not be repeated here. Attached Figure Description

[0030] Figure 1 This is a flowchart illustrating a speech recognition method provided in related technologies;

[0031] Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0032] Figure 3 A flowchart illustrating a speech recognition method provided in an embodiment of this application;

[0033] Figure 4 A flowchart illustrating another speech recognition method provided in an embodiment of this application;

[0034] Figure 5 A flowchart illustrating another speech recognition method provided in an embodiment of this application;

[0035] Figure 6 This is a schematic diagram illustrating the application of a speech recognition method provided in an embodiment of this application;

[0036] Figure 7 This is a schematic diagram of the structure of a speech recognition device provided in an embodiment of this application;

[0037] Figure 8 This is a schematic diagram of the structure of another electronic device provided in an embodiment of this application. Detailed Implementation

[0038] The terminology used in the implementation section of this application is for the purpose of explaining specific embodiments of this application only, and is not intended to limit this application.

[0039] Figure 1 This is a flowchart illustrating a speech recognition method provided in related technologies. For example... Figure 1 As shown, this method involves step ① inputting the audio data to be recognized into the audio processing module 11; step ② where the audio processing module 11 processes the audio data to obtain the processed audio data. Then, step ③ inputs the processed audio data into the ASR model 12 and outputs text. However, the audio processing of the audio data to be recognized may be affected by environmental noise, leading to lower accuracy. Furthermore, the ASR model requires a large amount of sample data for learning, and the continuous updating of entity words for specific domains (such as entity words in emerging fields or custom entity words) is usually limited. Therefore, the updated entity words have a relatively small impact on the training of the ASR model compared to the large amount of sample data, resulting in a poor ability of the ASR model to recognize entity words in specific domains. Therefore, the accuracy of speech recognition methods in related technologies for recognizing entity words in specific domains is low.

[0040] To address the aforementioned technical problems, this application provides a speech recognition method. The method includes acquiring audio data for target material; inputting the audio data into a speech recognition model to output a first text; and correcting entity words in the first text based on a first dataset of the target material to obtain a second text. The second text is the text translated from the audio data, and the first dataset is a set of multiple entity words from the target material. In other words, the audio data for the target material is input into a speech recognition model to obtain the first text, and then a standard dataset (i.e., the dataset of the target material) is used to correct the entity words in the first text to obtain the text translated from the audio data. It is evident that this application corrects the text output by the speech recognition model using the dataset of the target material. Since the target material can be data from a specific domain, the dataset of the target material can contain entity words from that specific domain. Therefore, this application can effectively assist the speech recognition model in accurately recognizing entity words from a specific domain, improving the accuracy of entity word recognition in that domain.

[0041] The execution subject of the speech recognition method provided in this application embodiment can be an electronic device or a component located in the electronic device (e.g., a chip, chip system, or processor, etc.). The following description uses an electronic device as the execution subject. For example, Figure 2 This is a schematic diagram of the structure of a voice recognition device provided in an embodiment of this application, as shown below. Figure 2 As shown, the electronic device 200 may include a speech recognition module 201 and a correction module 202. The speech recognition module 201 is used to recognize the audio data of the target material to obtain first text. Exemplarily, the speech recognition module 201 translates the audio data of the target material into first text using a speech recognition model. The correction module 202 is used to correct the entity words in the first text to obtain second text. Exemplarily, the correction module 202 compares the entity words of the first text with the entity words of a first dataset to correct the entity words of the first text to obtain the second text. The first dataset may be pre-obtained or determined in real time. In some embodiments, the electronic device 200 may further include a video recognition module 203. The video recognition module 203 is used to recognize the video data of the target material to obtain text information, and extract entity words from the text information to obtain a first dataset. For example, the video recognition module 203 is used to recognize the video data of the target material using an optical character recognition (OCR) model to obtain text information, and to extract entity words from the text information using a named-entity recognition (NER) model to obtain a first dataset.

[0042] The application scenarios of the speech recognition method provided in this application can include online and offline scenarios. Online scenarios include live streaming, sharing, and online conferencing. For domain-specific meetings, sharing sessions, or live streams, since the target material can be domain-specific, the dataset of the target material can contain domain-specific entity words. Therefore, the speech recognition method provided in this application can greatly improve the accuracy of recognizing professional terms, effectively enhance user experience, and reduce the cost of manual intervention. Offline scenarios include recorded video scenarios, online course scenarios, and supplementary subtitle scenarios for movies and TV shows. In these scenarios, since the target material can be domain-specific, the dataset of the target material can contain domain-specific entity words. Therefore, the speech recognition method provided in this application can effectively improve the transcription accuracy of professional terms.

[0043] The following describes in detail a speech recognition method provided by an embodiment of this application with reference to examples. Figure 3 This is a flowchart illustrating a speech recognition method provided in an embodiment of this application, as shown below. Figure 3 As shown, this method can be applied to an electronic device, which may include a display screen. The method may include steps S301-S303 (some of which are optional).

[0044] S301. Obtain audio data for the target material.

[0045] Target materials can be understood as audio data. Examples include speech drafts, presentation slides (Microsoft Office PowerPoint, PPT), and shared resources. The format of the target materials is not limited; for example, they can be in Word, PDF, text, or image formats. The target materials can vary depending on the application scenario. For example, in a speech scenario, the target materials can be speech drafts. In a conference scenario, the target materials can be PPT slides. In a live streaming scenario, the target materials can be shared resources used during the live stream.

[0046] Audio data related to target materials can be understood as audio data output based on the target materials; that is, audio data is the audio output by the speaker or attendee based on the target materials. For example, in a meeting scenario, it is the audio of the PPT content output by the attendees. In a speech scenario, it is the audio of the speech content output by the speaker based on the speech script.

[0047] The timing of acquiring voice data can be real-time. For example, audio data for a target document can be acquired in real-time through audio recording. For instance, in a meeting scenario, audio recording equipment can be activated at the start of the meeting, and all audio during the meeting can be recorded. Similarly, in a live streaming scenario, the audio and video of the live stream can be recorded during the broadcast.

[0048] As mentioned above, the recorded audio can be all the audio from a meeting or live stream. Therefore, the recorded audio can be analyzed and extracted to obtain audio data related to the target information. Irrelevant audio can be effectively filtered out, reducing the amount of computation for invalid data and saving power consumption.

[0049] S302. Input the audio data into the speech recognition model and output the first text.

[0050] Speech recognition models can be trained based on a reference number of sample audio data and the corresponding text. The reference number of sample audio data serves as the input data for the speech recognition model, and the corresponding text serves as the output data.

[0051] Based on the aforementioned speech recognition model, after inputting audio data, the first text of the audio data can be output. However, since training the speech recognition model requires a large amount of sample audio data, while the amount of audio data corresponding to domain-specific entity words or updated entity words is relatively small, the audio data corresponding to domain-specific entity words or updated entity words plays a smaller role in the training of the speech recognition model compared to the sample audio data, resulting in a poor ability of the speech recognition model to recognize domain-specific entity words. Therefore, it is necessary to correct the entity words in the text output by the speech recognition model, as detailed in the relevant description in S303.

[0052] S303. Based on the first dataset of the target data, correct the entity words in the first text to obtain the second text. The second text is the text translated from the audio data, and the first dataset is a set of multiple entity words of the target data.

[0053] The first dataset can include multiple entity words from the target data, or it can include multiple entity words from the target data and their corresponding audio features. Depending on the content of the first dataset, S303 can be implemented in different ways.

[0054] The first type involves a dataset containing multiple entity words from the target data. Accordingly, step S303 can be implemented as follows: Based on the order of entity words in the target data and the order of entity words in the first text, compare the similarity between each entity word in the target data and each entity word in the first text. If the similarity between an entity word in the target data and an entity word in the first text is greater than a threshold, and the similarity between the next entity word in the target data and the next entity word in the first text is less than a threshold, then the entity word in the first text is corrected to be an entity word from the target data. For example, the target data includes entity word 1, entity word 2, and entity word 3, where entity word 1 and entity word 2 are adjacent, and entity word 2 and entity word 3 are adjacent. The first text includes entity word 4, entity word 5, and entity word 6, where entity word 4 and entity word 5 are adjacent, and entity word 5 and entity word 6 are adjacent. If the similarity between entity word 1 and entity word 4 is greater than a threshold, and the similarity between entity word 2 and entity word 5 is less than a threshold, then entity word 5 in the first text is replaced with entity word 2. In this way, by comparing the similarity of adjacent entity words, the entity words in the first text are corrected, thereby achieving the purpose of correcting the entity words in the first text and obtaining a more accurate translated text of the speech data.

[0055] The second approach involves a first dataset containing audio features corresponding to multiple entity words in the target data. In another possible implementation, Figure 4 A flowchart illustrating another speech recognition method provided in this application embodiment is shown below. Figure 4As shown, S303 may include S3031 - S3034 (where some steps are optional). S3031: Compare the audio features corresponding to each entity word in the first data set with the audio features of each entity word in the first text. Exemplarily, the audio feature may be the pinyin of the entity word. For example, the first text includes the entity word "xianxing bijin method", and the first data set of the target material includes the entity word "linear approximation method". The audio feature (set of pinyin) of "linear approximation method" is "xian xing bi jin fa", and the audio feature (set of pinyin) of "xianxing bijin method" is "xian xing bi jin fa". S3032: When the audio features corresponding to the first entity word in the first data set are the same as those of the second entity word in the first text, determine the similarity between the first entity word and the second entity word. Continuing with the above example, compare the pinyin of "linear approximation method" and the pinyin of "xianxing bijin method". It can be seen that their pinyins are the same. At this time, compare the similarity between "linear approximation method" and "xianxing bijin method". S3033: When the similarity is greater than the threshold, correct the second entity word in the first text to the first entity word to obtain the second text. S3034: When the similarity is less than or equal to the threshold, determine that the first text is the second text. Assume that the threshold is 70%. Continuing with the above example, the similarity between "linear approximation method" and "xianxing bijin method" is 80%, which is greater than the threshold. Then replace "xianxing bijin method" in the first text with "linear approximation method" to obtain the second text. It can be seen that by comparing the audio features corresponding to each entity word in the first data set with the audio features of each entity word in the first text, when the audio features corresponding to the first entity word in the first data set are the same as those of the second entity word in the first text, if the similarity between the first entity word and the second entity word is greater than the threshold, correct the second entity word in the first text to the first entity word; if the similarity between the first entity word and the second entity word is less than or equal to the threshold, do not correct the second entity word in the first text. Thus, by correcting the text output by the speech recognition model, it helps the speech recognition model to translate audio data and improves the accuracy of the translated text of the speech data.

[0056] To further improve the accuracy of the translated text of the speech data, in some embodiments, Figure 5 is a schematic flowchart of another speech recognition method provided by an embodiment of the present application. As Figure 5 shown, before executing S303, a speech recognition method provided by an embodiment of the present application may further include: S304 - S308 (where some steps are optional).

[0057] S304: Obtain the video data of the target material.

[0058] The video data for the target materials can be recorded in real-time during the presentation of the target materials, or it can be video data extracted from the target materials beforehand. These two scenarios can be tailored to different applications. For example, in scenarios with high real-time requirements (such as live streaming or sharing scenarios), the video data of the target materials can be recorded in real-time during the presentation. On the other hand, in scenarios with lower real-time requirements (such as conference scenarios), if the target materials are entered into the conference system during the conference preparation stage, the target materials can be analyzed and their video data extracted, such as by taking photos or recording videos of each page of the target materials.

[0059] The video data of the target material can be understood as a video recorded for the target material. The aforementioned video data can be a video stream composed of multiple images arranged in time. Therefore, the video data can include multiple images. Since the multiple images may include duplicate images and images without text information, it is necessary to filter out duplicate images and images without text information from the multiple images during the construction of the first dataset, thereby reducing the computational load. Therefore, after executing S304, the speech recognition method provided in this application embodiment may further include: S307, filtering out multiple key images from the multiple images according to time. Wherein, S307... Figure 5 (Not shown in the image). Multiple key images can be understood as the necessary images required to create the first dataset. The selection of multiple key images can be done as follows: In one example, S307 can be implemented as follows: Select multiple non-repeating images from multiple images according to time as multiple key images. That is, filter out duplicate images from multiple images according to time, leaving multiple images as multiple key images. In this way, by selecting multiple non-repeating images from the video data to create the first dataset, unnecessary computation is reduced and power consumption is saved while ensuring the accuracy of the first dataset. In another example, S307 can be implemented as follows: Select multiple images containing text information from multiple images according to time as multiple key images. That is, filter out images that do not contain text information from multiple images according to time, leaving multiple images containing text information as multiple key images. In this way, by selecting multiple images containing text information from the video data to create the first dataset, unnecessary computation is reduced and power consumption is saved while ensuring the accuracy of the first dataset.

[0060] Of course, the two examples above can also be combined to obtain multiple key images. For example, in another example, S307 can be implemented as follows: filtering multiple non-repeating images from multiple images according to time, and then filtering multiple images containing text information from these multiple images according to time as multiple key images. In this way, by filtering the video data to select valid data as the data for creating the first dataset, invalid calculations are reduced and power consumption is saved while ensuring the accuracy of the first dataset. Therefore, no specific limitations are made in the embodiments of this application.

[0061] S305. Input the video data into the optical character recognition model and output the third text.

[0062] The principle of optical character recognition (OCR) models is to use optical and computer technologies to extract text printed or written on paper into text form and convert it into a format that computers can accept and understand. In other words, inputting video data into an OCR model will output third-party text.

[0063] Continuing with the above example, S305 can be implemented as follows: inputting multiple key images into the optical character recognition model and outputting third text. Multiple key images are obtained through S307 above, and these key images are input into the optical character recognition model to obtain the third text. In this way, by extracting multiple key images from multiple images as input to the optical character recognition model, the ineffective calculations of the optical character recognition model can be effectively simplified, and the calculation speed and efficiency of the optical character recognition model can be effectively improved.

[0064] S306. Extract each entity word from the third text to obtain the first dataset.

[0065] After obtaining the third text, entity words can be extracted from it. In one possible implementation, S306 can be implemented as follows: input the third text into an entity recognition model and output the first dataset. The entity recognition model can be a named entity recognition (NER) model, which can identify entities with specific meanings in the text, such as names of people, places, organizations, and proper nouns. Therefore, inputting the third text into the NER model yields the first dataset.

[0066] In this application, the first dataset is obtained through video data of the target material. In this way, the video data of the target material is used as the correction reference data for the output text of the speech recognition model, which effectively improves the accuracy of recognition and reduces the cost of manual proofreading.

[0067] In summary, the target data mentioned above can include data from a specific domain. This means that the target data contains entity words from that specific domain. Therefore, the first dataset obtained based on the target data, and the text output by the speech recognition model corrected based on the first dataset, can effectively assist the speech recognition model in accurately recognizing entity words from that specific domain, thereby improving the accuracy of entity word recognition.

[0068] S308, Display the second text.

[0069] The second text can be understood as subtitles for the audio data. Therefore, after obtaining the second text, it can be displayed on the screen of an electronic device. This allows users and viewers to view the subtitles on the screen, thus enhancing the experience.

[0070] In practical applications, Figure 6 This is a schematic diagram illustrating the application of a speech recognition method provided in an embodiment of this application. For example... Figure 6As shown, assuming an academic lecture (internal datasets) scenario, the audio data includes audio file A and audio file B, and the video data includes video file C. By inputting audio file A into a speech recognition model, text file 1 is output; by inputting audio file B into the speech recognition model, text file 2 is output. Text file 1 contains the statement, "Here, even without any vivid knowledge of cold body tones, we have found a solution much better than existing methods." Text file 2 contains the statement, "We can use this constrained approximation method to solve this Pareto optimization problem." By performing repetition detection and filtering on video file C, multiple key images are obtained, and these key images are input into an OCR model, outputting text file 3. Text file 3 includes statements such as, "Proposed four equivalent objective function construction methods including warm start and cold start," "Efficient Pareto Frontier Search Algorithm," "Feasibility Verification and Example Testing of All Methods," "Linear Approximation Method," and "...". Then, text file 3 is input into a NER model to obtain the first dataset, which may include entity words such as "warm start," "equivalent objective function," "construction method," "cold start," "linear approximation method," and "...". Text files 1 and 2 are compared with the first dataset, and the entity words in text files 1 and 2 are adjusted according to the comparison results. For example, the entity word "cold tone" in text file 1 does not match "cold start" in the first dataset, so "cold tone" in text file 1 is changed to "cold start", forming text file 4. Similarly, the entity word "limited line approximation method" in text file 2 does not match "linear approximation method" in the first dataset, so "limited line approximation method" in text file 1 is changed to "linear approximation method", forming text file 5. Text files 4 and 5 are then displayed on the screen. It is evident that the first dataset obtained based on the target data, and the text output by the speech recognition model corrected based on the first dataset, can effectively assist the speech recognition model in accurately recognizing entity words in specific domains, improving the accuracy of entity word recognition in specific domains.

[0071] Experiments showed that, using three videos from the Huawei Strategy and Technology Workshop (STW) as the test dataset, the word error rate (WER) of the speech recognition results obtained using the speech recognition method provided in this application was lower than that of the speech recognition results obtained using related technologies. Table 1 shows a comparison of the WER metrics between this application and related technologies.

[0072] Table 1

[0073] Dataset Related technologies This application Dataset-1 15.24 15.19 Dataset-2 16.21 16.12 Dataset-3 9.14 9.06

[0074] As shown in Table 1, the WER index of this application is lower than that of related technologies. This demonstrates that the speech recognition method provided in this application has a lower text error rate and higher recognition accuracy.

[0075] For example, the entity f1 score of the speech recognition method provided in this application is higher than that of related technologies. The entity f1 score is used to measure the accuracy of entity word recognition. Table 2 shows a comparison of the entity f1 scores of this application and related technologies.

[0076] Table 2

[0077] Dataset Related technologies This application Dataset-1 70.59 73.56 Dataset-2 87.65 91.02 Dataset-3 42.86 84.21

[0078] As shown in Table 1, the entity f1 score of this application is higher than that of related technologies. This demonstrates that the speech recognition method provided in this application achieves higher accuracy in recognizing entity words.

[0079] As can be seen, by outputting text from audio data based on a traditional speech recognition model, and then correcting the text output by the speech recognition model using a dataset of target data, since the target data can be data from a specific domain, the dataset of the target data can contain entity words from that specific domain. Therefore, this application can effectively assist the speech recognition model in accurately recognizing entity words from a specific domain and improve the recognition accuracy of entity words from that specific domain.

[0080] The above describes a speech recognition method provided by embodiments of this application. Corresponding to the above method, embodiments of this application also provide a speech recognition device. This device is applied to an electronic device. The device is used to... Figure 7 Each module shown performs the above... Figure 3 The speech recognition methods performed by electronic devices in China. For example... Figure 7 As shown, the speech recognition device 500 provided in this application embodiment includes the following modules.

[0081] The acquisition module 501 is used to acquire audio data for the target material.

[0082] The output module 502 is used to input the audio data into the speech recognition model and output the first text.

[0083] The correction module 503 is used to correct the entity words in the first text according to the first dataset of the target data to obtain the second text, wherein the second text is the text translated from the audio data, and the first dataset is a set of multiple entity words of the target data.

[0084] In this way, by inputting the audio data of the target material into the speech recognition model to obtain the first text, and then using a standard dataset (i.e., the dataset of the target material) to correct the entity words in the first text, the translated text of the audio data is obtained. It can be seen that this application corrects the text output by the speech recognition model by using the dataset of the target material. Since the target material can be data from a specific domain, the dataset of the target material can contain entity words from that specific domain. Therefore, this application can effectively assist the speech recognition model in accurately recognizing entity words from a specific domain, thereby improving the accuracy of entity word recognition in that domain.

[0085] In one possible implementation, the first dataset also includes audio features corresponding to multiple entity words; the correction module 503 is used to: compare the audio features corresponding to each entity word in the first dataset with the audio features of each entity word in the first text; if the audio features corresponding to the first entity word in the first dataset and the second entity word in the first text are the same, determine the similarity between the first entity word and the second entity word; if the similarity is greater than a threshold, correct the second entity word in the first text to the first entity word to obtain the second text.

[0086] In one possible implementation, the correction module 503 is used to determine the first text as the second text if the similarity is less than or equal to a threshold.

[0087] In one possible implementation, the acquisition module 501 is used to acquire video data of the target material; the input module 504 is used to input the video data into the optical character recognition model and output the third text; and the extraction module 505 is used to extract each entity word in the third text to obtain the first dataset.

[0088] In one possible implementation, the video data includes multiple images; the device 500 also includes: a filtering module 506 for filtering multiple key images from the multiple images according to time; and an input module 504 for inputting the multiple key images into an optical character recognition model and outputting third text.

[0089] In one possible implementation, the filtering module 506 is used to filter multiple non-repeating images from multiple images according to time as multiple key images.

[0090] In one possible implementation, the filtering module 506 is used to filter multiple images containing text information from multiple images according to time as multiple key images.

[0091] In one possible implementation, the extraction module 505 is used to input the third text into the entity recognition model and output the first dataset.

[0092] It should be understood that the above Figure 7 The beneficial effects that the provided device possesses in performing its function are... Figure 3 The provided speech recognition methods offer the same beneficial effects, which will not be elaborated upon here. Additionally, Figure 7 The provided device, in implementing its functions, is only illustrated by the division of the above-described functional modules. In practical applications, the functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the device and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation processes are detailed in the method embodiments, and will not be repeated here.

[0093] See Figure 8 , Figure 8 A schematic diagram of the structure of an exemplary electronic device 600 of this application is shown. The electronic device 600 includes at least one processor 601, a memory 603, and at least one network interface 604.

[0094] Processor 601 may be, for example, a general-purpose central processing unit (CPU), a digital signal processor (DSP), a network processor (NP), a GPU, a neural-network processing unit (NPU), a data processing unit (DPU), a microprocessor, or one or more integrated circuits or application-specific integrated circuits (ASICs), programmable logic devices (PLDs), other general-purpose processors or other programmable logic devices, discrete gates, transistor logic devices, discrete hardware components, or any combination thereof for implementing the scheme of this application. A PLD may be, for example, a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. A general-purpose processor may be a microprocessor or any conventional processor. It is worth noting that the processor may be a processor supporting an advanced reduced instruction set machine (RISC) machine (ARM) architecture. It can implement or execute various logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0095] Optionally, the electronic device 600 also includes a bus 602. The bus 602 is used to transmit information between the various components of the electronic device 600. The bus 602 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus 602 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 The symbol is represented by only one line, but this does not mean that there is only one bus or one type of bus.

[0096] Memory 603 may be, for example, volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which is used as an external cache.

[0097] By way of example, but not limitation, many forms of ROM and RAM are available. For example, ROM is a compact disc read-only memory (CD-ROM). RAM includes, but is not limited to, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0098] The memory 603 can also be other types of storage devices capable of storing static information and instructions. Alternatively, it can be other types of dynamic storage devices capable of storing information and instructions. It can also be other optical disc storage, optical disk storage (including compressed optical discs, laser discs, optical discs, digital versatile optical discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. The memory 603 may exist independently, for example, and be connected to the processor 601 via bus 602. The memory 603 may also be integrated with the processor 601.

[0099] Network interface 604 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), or wireless local area network (WLAN). Network interface 604 can include wired network interfaces and wireless network interfaces. Specifically, network interface 604 can be an Ethernet interface, such as Fast Ethernet (FE), Gigabit Ethernet (GE), Asynchronous Transfer Mode (ATM), WLAN, cellular network, or combinations thereof. The Ethernet interface can be an optical interface, an electrical interface, or a combination thereof. In some embodiments of this application, network interface 604 can be used by electronic device 600 to communicate with other devices.

[0100] In specific implementations, as some embodiments, processor 601 may include one or more CPUs, such as Figure 8 The CPU0 and CPU1 shown are examples of processors. Each of these processors can be a single-core processor or a multi-core processor. A processor here can refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0101] In specific implementations, as some embodiments, the electronic device 600 may include multiple processors, such as... Figure 8 The processors 601 and 605 are shown in the diagram. Each of these processors may be a single-core processor or a multi-core processor. A processor here may refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0102] In some embodiments, memory 603 is used to store program instructions 610 for executing the scheme of this application, and processor 601 can execute the program instructions 610 stored in memory 603. That is, electronic device 600 can implement the method provided in the method embodiment through processor 601 and program instructions 610 in memory 603, i.e. Figure 3 The method executed. Program instructions 610 may include one or more software modules. Optionally, processor 601 itself may also store program instructions for executing the scheme of this application.

[0103] In specific implementation, the electronic device 600 of this application can correspond to a first network element device for executing the above method. The processor 601 in the electronic device 600 reads the instructions in the memory 603, causing... Figure 8 The electronic device 600 shown is capable of performing all or part of the steps in the method embodiments.

[0104] Electronic device 600 can also correspond to the above. Figure 5 The device shown, Figure 5 Each functional module in the illustrated device is implemented using software from electronic device 600. In other words, Figure 5 The device shown includes functional modules generated by the processor 601 of the electronic device 600 after reading the program instructions 610 stored in the memory 603.

[0105] in, Figure 3 Each step of the method shown is implemented through integrated logic circuits in the hardware of the processor of the electronic device 600 or through instructions in software form. The steps of the method embodiments disclosed in this application can be directly implemented by the hardware processor, or implemented by a combination of hardware and software modules in the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other storage media mature in the art. Since this storage medium is located in memory, the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method embodiments; to avoid repetition, they will not be described in detail here.

[0106] In an exemplary embodiment, an electronic device is provided, the electronic device including a processor coupled to a memory; the memory stores at least one instruction, the at least one instruction being loaded and executed by the processor to cause the electronic device to perform... Figure 3 The speech recognition method in [the context of the text].

[0107] In an exemplary embodiment, a computer program (product) is provided, comprising: computer program code, which, when executed by a computer, causes the computer to perform... Figure 3 The speech recognition method in [the context of the text].

[0108] In an exemplary embodiment, a computer-readable storage medium is provided that stores a program or instructions, which, when executed on a computer, cause the computer to perform the aforementioned actions. Figure 3 The speech recognition method in [the context of the text].

[0109] In an exemplary embodiment, a chip is provided, including a processor for recalling and executing instructions stored in memory, causing a computer with the chip installed to perform... Figure 3 The speech recognition method in [the context of the text].

[0110] In an exemplary embodiment, another chip is provided, including: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected via internal interconnection paths. The processor is used to execute code in the memory. When the code is executed, a computer with the chip installed performs... Figure 3 The speech recognition method in [the context of the text].

[0111] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive).

[0112] It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the setting results involved in this application were obtained with full authorization.

[0113] Those skilled in the art will recognize that the method steps and modules described in conjunction with the embodiments disclosed herein can be implemented in software, hardware, firmware, or any combination thereof. To clearly illustrate the interchangeability of hardware and software, the steps and components of each embodiment have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0114] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0115] When implemented using software, it can be implemented wholly or partially as a computer program product. This computer program product includes one or more computer program instructions. As an example, the methods of this application embodiment can be described in the context of machine-executable instructions, such as program modules that execute on a device on a real or virtual processor of the target. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, etc., which perform specific tasks or implement specific abstract data structures. In various embodiments, the functionality of program modules can be combined or divided among the described program modules. The machine-executable instructions for the program modules can execute within a local or distributed device. In a distributed device, the program modules can reside on both local and remote storage media.

[0116] Computer program code used to implement the methods of the embodiments of this application may be written in one or more programming languages. This computer program code may be provided to the processor of a determined device of a general-purpose computer, a special-purpose computer, or other programmable agent node, such that when executed by the computer or other programmable agent node, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a computer, partially on a computer, as a standalone software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server.

[0117] In the context of the embodiments of this application, computer program code or related data may be carried by any suitable carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, etc.

[0118] Examples of signals may include electrical, optical, radio, sound, or other forms of propagation signals, such as carrier waves, infrared signals, etc.

[0119] A machine-readable medium can be any tangible medium that contains or stores programs for or relating to an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. More detailed examples of machine-readable storage media include electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0120] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be found in the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0121] In the embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through some interfaces, devices, or modules, or they may be electrical, mechanical, or other forms of connection.

[0122] The modules described as separate components may or may not be physically separate. Similarly, the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0123] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0124] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0125] In this application, the terms "first," "second," etc., are used to distinguish identical or similar items that have substantially the same function and purpose. It should be understood that there is no logical or temporal dependency between "first," "second," and "nth," nor does it limit the quantity or order of execution. It should also be understood that although the following description uses the terms "first," "second," etc., to describe various elements, these elements should not be limited by the terms. These terms are merely used to distinguish one element from another. For example, without departing from the various examples described, a first image can be referred to as a second image, and similarly, a second image can be referred to as a first image. Both the first image and the second image can be images, and in some cases, they can be separate and distinct images.

[0126] It should also be understood that, in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0127] In this application, the term "at least one" means one or more, and the term "multiple" means two or more. For example, multiple second messages refer to two or more second messages. The terms "system" and "network" are often used interchangeably in this document.

[0128] It should be understood that the terminology used in the description of the various examples herein is for the purpose of describing particular examples only and is not intended to be limiting. As used in the description of the various examples and the appended claims, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0129] It should also be understood that the term "and / or" as used herein refers to and covers any and all possible combinations of one or more of the associated listed items. The term "and / or" describes an association between related objects, indicating that three relationships can exist; for example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " in this application generally indicates that the preceding and following related objects are in an "or" relationship.

[0130] It should also be understood that the term “comprising” (also referred to as “includes”, “including”, “comprises” and / or “comprising”) as used in this specification specifies the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0131] It should also be understood that the terms “if” and “if” can be interpreted as meaning “when” or “upon”, or “in response to determination” or “in response to detection”. Similarly, depending on the context, the phrases “if determination…” or “if detection [the stated condition or event]” can be interpreted as meaning “when determination…”, or “in response to determination…”, or “when detection [the stated condition or event]” or “in response to detection [the stated condition or event]”.

[0132] It should be understood that determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information.

[0133] It should also be understood that the phrases "an embodiment," "an embodiment," and "a possible implementation" used throughout the specification mean that a specific feature, structure, or characteristic related to an embodiment or implementation is included in at least one embodiment of this application. Therefore, the phrases "in an embodiment," "an embodiment," or "a possible implementation" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

Claims

1. A speech recognition method, characterized in that, The method includes: Acquire audio data for the target material; The audio data is input into the speech recognition model, and the first text is output. Based on the first dataset of the target data, the entity words in the first text are corrected to obtain the second text, which is the text translated from the audio data. The first dataset is a set of multiple entity words of the target data.

2. The method according to claim 1, characterized in that, The first dataset also includes audio features corresponding to the plurality of entity words; the step of correcting the entity words in the first text based on the first dataset of the target data to obtain the second text includes: Compare the audio features corresponding to each entity word in the first dataset with the audio features of each entity word in the first text; If the audio features corresponding to the first entity word in the first dataset and the second entity word in the first text are the same, determine the similarity between the first entity word and the second entity word. If the similarity is greater than the threshold, the second entity word in the first text is corrected to the first entity word to obtain the second text.

3. The method according to claim 2, characterized in that, Also includes: If the similarity is less than or equal to the threshold, the first text is determined to be the second text.

4. The method according to any one of claims 1-3, characterized in that, Before obtaining the second text by correcting entity words in the first text based on the first dataset of the target data, the method further includes: Acquire the video data of the target material; The video data is input into the optical character recognition model, and the third text is output. Extract each entity word from the third text to obtain the first dataset.

5. The method according to claim 4, characterized in that, The video data includes multiple images; after acquiring the video data of the target material, it also includes: Select multiple key images from the multiple images according to time; The step of inputting the video data into the optical character recognition model and outputting third text includes: The multiple key images are input into the optical character recognition model, and the third text is output.

6. The method according to claim 5, characterized in that, The step of filtering multiple key images from the multiple images according to time includes: Multiple unique images are selected from the multiple images according to time as the multiple key images.

7. The method according to claim 5, characterized in that, The step of filtering multiple key images from the multiple images according to time includes: Multiple images containing text information are selected from the multiple images according to time as the multiple key images.

8. The method according to any one of claims 4-7, characterized in that, The step of extracting entity words from the third text to obtain the first dataset includes: The third text is input into the entity recognition model, and the first dataset is output.

9. A voice recognition device, characterized in that, The device includes: The acquisition module is used to acquire audio data for the target material; The output module is used to input the audio data into the speech recognition model and output the first text. The correction module is used to correct entity words in the first text based on the first dataset of the target data to obtain a second text, wherein the second text is the text translated from the audio data, and the first dataset is a set of multiple entity words of the target data.

10. An electronic device, characterized in that, The device includes a processor coupled to a memory; the memory stores at least one instruction, which is loaded and executed by the processor to cause the electronic device to perform the method of any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction, which is loaded and executed by a processor to implement the method as described in any one of claims 1-8.

12. A computer program product, characterized in that, The computer program product includes a computer program / instruction that is executed by a processor to cause a computer to perform the method described in any one of claims 1-8.

Images