A railway dispatching speech annotation method and system

Abstract

The present invention provides a railway dispatching speech annotation method and system, which comprises the following steps: firstly obtaining a railway dispatching speech file to be annotated, performing pre-annotation processing on the railway dispatching speech file to be annotated based on an artificial intelligence speech annotation algorithm to obtain a pre-annotated file; displaying the railway dispatching speech file to be annotated in an interface image in the form of a speech waveform; arranging a plurality of pre-annotated data in the pre-annotated file according to a preset sorting rule and displaying them in the interface image; adding a semi-transparent mask to a corresponding time period of the speech waveform based on the speech start and end time; correcting the pre-annotated file based on the semi-transparent mask and a preset correction strategy to obtain a labeled file; and outputting the labeled file and the railway dispatching speech file to be annotated as an annotation result.

Description

Technical Field

[0001] The invention relates to a speech annotation method, and in particular to a railway dispatching speech annotation method and system. Background Art

[0002] Railway transportation dispatching is the command center of daily railway transportation organization. In order to give full play to the traction role of transportation dispatching command in the entire transportation production process, it is necessary to improve the comprehensive management and analysis capabilities of the dispatching office and build a comprehensive management and analysis system for the dispatching office. The importance of railway dispatching requires that the comprehensive management and analysis system of the dispatching office must have extremely high analytical capabilities. To improve the analytical capabilities, it is necessary to annotate a railway dispatching corpus with rich and accurate information as the research basis.

[0003] The common speech annotation systems based on neural network models on the market have a high accuracy rate for standardized speech files. However, the dispatcher's verbal instructions are very serious, and the dispatcher speaks at a fast speed, with long texts, overlapping speeches, and few silent segments. In addition, the recordings of dispatches by most railway bureaus across the country have low signal-to-noise ratios, and are subject to strong noise, strong interference, multiple dialects, and multiple terms, which makes it difficult for such annotation systems to achieve accurate segmentation and text annotation of dispatching speech.

[0004] In addition, the railway needs to separate the dispatcher and the callee, and the conversations between two or more dispatchers are stored together in a single channel. The general voiceprint recognition model cannot complete the labeling of the identities of multiple dispatchers in the dispatch recording. Railway dispatch voice also contains a lot of environmental sounds that are not dispatch dialogue content, such as whistle sounds, tunnel shuttle sounds, thunder, rain, etc. Although these environmental sounds do not belong to the dispatch content, they can represent the situation at the time, such as whether it is passing through a tunnel, whether it is accompanied by rain, etc. This part of the data is of great value in highly restoring the dispatch scenario, so it is necessary to label the environmental sounds, but the existing voice labeling system lacks the labeling of this type of information.

[0005] In order to solve the above problems, people have been seeking an ideal technical solution. Summary of the invention

[0006] The purpose of the present invention is to address the deficiencies of the prior art and thus provide a method and system for labeling railway dispatching speech, which can achieve accurate segmentation and labeling, produce high-quality railway dispatching corpus with rich labeling information, and label environmental sounds at the same time.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is: a method for marking railway dispatching speech, comprising the following steps:

[0008] Acquire a railway dispatching voice file to be annotated, and perform pre-annotation processing on the railway dispatching voice file to be annotated based on an artificial intelligence voice annotation algorithm to obtain a pre-annotated file, wherein the pre-annotated file includes a plurality of pre-annotated data and a voice start and end time associated with each pre-annotated data;

[0009] The railway dispatching voice file to be annotated is displayed in the interface image in the form of a voice waveform; a plurality of pre-annotated data in the pre-annotated file are arranged and displayed in the interface image according to a preset sorting rule; a semi-transparent mask is added to the corresponding time period of the voice waveform based on the voice start and end time, and each semi-transparent mask is associated with a corresponding voice start and end time; and a preset mouse event is associated with each semi-transparent mask and each pre-annotated data;

[0010] Correct the pre-annotated file based on the semi-transparent mask and the preset correction strategy to obtain the annotated file:

[0011] When it is detected that the waveform start and end time corresponding to the semi-transparent mask changes, the waveform start and end time corresponding to the semi-transparent mask after the change is obtained, and the waveform start and end time is used to update and correct the speech start and end time associated with the semi-transparent mask;

[0012] When a preset mouse event is detected for the semi-transparent mask, the corresponding voice segment is searched for and played in the to-be-annotated railway dispatching voice file based on the voice start and end time associated with the semi-transparent mask, and after receiving manual input data, the pre-annotated data associated with the semi-transparent mask is updated with the manual input data;

[0013] or when detecting that the pre-annotated data has a preset mouse event, searching for a corresponding voice segment in the to-be-annotated railway dispatching voice file based on the voice start and end time associated with the pre-annotated data for playback, and after receiving manual input data, updating the pre-annotated data with the manual input data;

[0014] The annotated file and the railway dispatching voice file to be annotated are output as the annotation results.

[0015] Based on the above, during the manual correction of the pre-annotated file based on the semi-transparent mask and the preset correction strategy, it is also possible to detect whether there is a semi-transparent mask addition operation on the speech waveform. If so, the waveform start and end time corresponding to the newly added semi-transparent mask is obtained as the newly added speech start and end time, and the newly added speech start and end time is added to the pre-annotated file, and the newly added speech start and end time is associated and bound with the newly added semi-transparent mask, and the newly added speech start and end time is arranged and displayed in the interface image according to the preset sorting rules, and the preset mouse event is associated with the newly added semi-transparent mask;

[0016] It is further detected whether there is a preset mouse event for the newly added semi-transparent mask. If so, the corresponding voice segment is searched in the railway scheduling voice file to be annotated based on the voice start and end time associated with the newly added semi-transparent mask for playback. After receiving the manually input data, the manually input data is associated and bound with the newly added voice start and end time as the newly added pre-annotated data, and added to the pre-annotated file. The newly added pre-annotated data is displayed in the interface image, and the preset mouse event is associated with the newly added pre-annotated data.

[0017] Based on the above, the conditions for the change of the start and end time of the waveform corresponding to the semi-transparent mask are that there is an operation of dragging the entire semi-transparent mask, there is an operation of dragging the left boundary line of the semi-transparent mask, and / or there is an operation of dragging the right boundary line of the semi-transparent mask.

[0018] Based on the above, when there is a translucent mask adding operation, a translucent mask overall dragging operation, a translucent mask left boundary line dragging operation or a translucent mask right boundary line dragging operation, it is also necessary to determine whether the current operation is valid based on the translucent mask non-overlapping strategy.

[0019] Based on the above, the steps for obtaining the pre-marked file are as follows:

[0020] For the railway dispatching voice files to be annotated, a deep learning sound source separation model is used to separate the human voice signal and the ambient sound signal to obtain human voice signal segments and ambient sound signal segments;

[0021] For the vocal signal segments, the vocal signal segments are logically segmented using the vocal separation clustering model to obtain the vocal logical segments; the vocal logical segments are transcribed into text using the speech recognition model to obtain text annotation information; and the vocal logical segments are clustered based on the dispatcher identity, and the same virtual dispatcher identifier is annotated for multiple vocal logical segments belonging to the same dispatcher, and then the virtual dispatcher identifier is identified and associated with the dispatcher identity in the vocal voiceprint library using the voiceprint recognition model to obtain the dispatcher identity annotation information;

[0022] For the ambient sound signal fragments, the ambient sound separation clustering model is used to logically segment the ambient sound signal fragments to obtain ambient sound logical fragments; the ambient sound logical fragments are clustered based on the ambient sound categories, and multiple ambient sound logical fragments belonging to the same ambient sound are marked with the same virtual ambient sound identifier; the ambient sound category recognition model is used to identify and associate the virtual ambient sound identifier with the ambient sound identity in the ambient sound voiceprint library to obtain the ambient sound labeling information;

[0023] Based on a preset integration strategy, the text annotation information of the human voice logic segment, the dispatcher identity annotation information and the ambient sound category annotation information of the ambient sound logic segment are integrated to obtain a number of pre-annotated data, and finally a pre-annotated file containing the pre-annotated data and the voice start and end time of the pre-annotated data is output.

[0024] Based on the above, the steps for obtaining the pre-marked file are as follows:

[0025] For the railway dispatching voice files to be annotated, a deep learning sound source separation model is used to separate the human voice signal and the ambient sound signal to obtain human voice signal segments and ambient sound signal segments;

[0026] For the vocal signal segments, the vocal signal segments are logically segmented using the vocal separation clustering model to obtain the vocal logical segments; the vocal logical segments are transcribed into text using the speech recognition model to obtain text annotation information; and the vocal logical segments are clustered based on the dispatcher identity, and the same virtual dispatcher identifier is annotated for multiple vocal logical segments belonging to the same dispatcher, and then the virtual dispatcher identifier is identified and associated with the dispatcher identity in the vocal voiceprint library using the voiceprint recognition model to obtain the dispatcher identity annotation information;

[0027] For the ambient sound signal fragments, the ambient sound separation clustering model is used to logically segment the ambient sound signal fragments to obtain ambient sound logical fragments; the ambient sound logical fragments are clustered based on the ambient sound categories, and multiple ambient sound logical fragments belonging to the same ambient sound are marked with the same virtual ambient sound identifier; the ambient sound category recognition model is used to identify and associate the virtual ambient sound identifier with the ambient sound identity in the ambient sound voiceprint library to obtain the ambient sound labeling information;

[0028] Based on a preset integration strategy, the text annotation information of the human voice logic segment, the dispatcher identity annotation information and the ambient sound category annotation information of the ambient sound logic segment are integrated to obtain a number of pre-annotated data, and finally a pre-annotated file containing the pre-annotated data and the voice start and end time of the pre-annotated data is output.

[0029] Based on the above, the preset integration strategy is executed:

[0030] Traversing the human voice logical segments, for each human voice logical segment, determining whether there is an ambient sound logical segment whose speech start time is greater than or equal to the speech start time of the human voice logical segment;

[0031] If so, it is further determined whether the voice ending time of the ambient sound logic segment is greater than or equal to the voice ending time of the human voice logic segment; if so, it is determined that the human voice logic segment is within the ambient sound logic segment, and the ambient sound logic segment is subjected to secondary logical segmentation according to the voice start and end time of the human voice logic segment; otherwise, it is determined that the human voice logic segment partially overlaps with the ambient sound logic segment, and the ambient sound logic segment is subjected to secondary logical segmentation according to the voice start time of the human voice logic segment;

[0032] If it does not exist, further determine whether there is an ambient sound logic segment whose voice termination time is less than or equal to the voice termination time of the human voice logic segment; if it does exist, determine that the ambient sound logic segment is located within the human voice logic segment and do not perform any operation; otherwise, determine that the human voice logic segment and the ambient sound logic segment partially overlap, and perform a secondary logical segmentation on the ambient sound logic segment according to the voice termination time of the human voice logic segment;

[0033] Traverse each voice start and end time to determine whether there are both human voice logic segments and ambient sound logic segments. If so, integrate the dispatcher identity annotation information, text annotation information, and ambient sound category annotation information corresponding to the current voice start and end time to form the pre-annotated data of the current voice start and end time; if not, use the dispatcher identity annotation information and text annotation information, or ambient sound category annotation information corresponding to the current voice start and end time as the pre-annotated data of the current voice start and end time;

[0034] Arrange the pre-annotated data of each speech start and end time in the order of the speech start and end time.

[0035] The present invention also provides a railway dispatching speech annotation system, comprising:

[0036] A labeling task acquisition module, used to acquire a labeling task, wherein the labeling task includes a railway dispatching voice file to be labeled;

[0037] An algorithm-assisted pre-annotation module is used to perform pre-annotation processing on the railway dispatching voice file to be annotated based on an artificial intelligence voice annotation algorithm to obtain a pre-annotated file, wherein the pre-annotated file includes a plurality of pre-annotated data and the voice start and end time associated with each pre-annotated data;

[0038] The marking task display module is used to display the railway dispatching voice file to be marked in the interface image in the form of voice waveform; arrange and display several pre-marked data in the pre-marked file in the interface image according to the preset sorting rules; add a semi-transparent mask to the corresponding time period of the voice waveform based on the voice start and end time, and each semi-transparent mask is correspondingly associated with one of the voice start and end time; and associate a preset mouse event with each semi-transparent mask and each pre-marked data;

[0039] The visual correction module is used to correct the pre-annotated file based on the semi-transparent mask and the preset correction strategy to obtain the annotated file:

[0040] When it is detected that the waveform start and end time corresponding to the semi-transparent mask changes, the waveform start and end time corresponding to the semi-transparent mask after the change is obtained, and the waveform start and end time is used to update and correct the speech start and end time associated with the semi-transparent mask;

[0041] When a preset mouse event is detected for the semi-transparent mask, the corresponding voice segment is searched for and played in the to-be-annotated railway dispatching voice file based on the voice start and end time associated with the semi-transparent mask, and after receiving manual input data, the pre-annotated data associated with the semi-transparent mask is updated with the manual input data;

[0042] or when detecting that the pre-annotated data has a preset mouse event, searching for a corresponding voice segment in the to-be-annotated railway dispatching voice file based on the voice start and end time associated with the pre-annotated data for playback, and after receiving manual input data, updating the pre-annotated data with the manual input data;

[0043] The labeling task result output module is used to output the labeling file and the railway dispatching voice file to be labeled as the labeling result.

[0044] Furthermore, it also includes a voice signal preprocessing module, an audit module, a sampling module, a task management module and a role management module;

[0045] The role management module is used to set different roles and assign different processing permissions to different roles; the roles include task administrators, labelers, reviewers, and sampling personnel;

[0046] The speech signal preprocessing module is used to perform speech detection processing, environmental sound suppression processing and data enhancement processing on the speech signal of the railway dispatching speech file to be marked;

[0047] The task management module is used for the task administrator to import the railway dispatching voice files to be annotated and export the valid corpus;

[0048] The audit module is used for auditors to receive tasks to be audited and conduct visual audit work, and after the audit is passed, the tasks to be audited and the audit results are converted into tasks to be sampled for release, and when the audit fails, the audit opinions are returned, and the tasks to be audited are returned to the annotators for re-annotation, wherein the tasks to be audited are the annotated results;

[0049] The sampling inspection module is used for sampling inspection personnel to receive the tasks to be sampled, conduct visual review work, and mark the tasks to be sampled as qualified for sampling inspection after the review is passed, and the marking results generated by the tasks are marked as valid corpus.

[0050] The present invention also provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the aforementioned railway dispatching speech annotation method when executing the program.

[0051] The present invention has outstanding substantive features and significant progress compared to the prior art. Specifically,

[0052] (1) Assisted annotation through pre-annotation: The railway dispatching voice files to be annotated are pre-annotated using an artificial intelligence speech annotation algorithm to obtain pre-annotated files. The pre-annotated files are manually corrected without the need for manual annotation. The efficiency of railway dispatching corpus production is greatly improved through automatic pre-annotation and manual correction.

[0053] (2) Realize visual and precise segmentation and labeling: The railway dispatching voice file to be labeled is presented in waveform form. A semi-transparent mask is added to the corresponding time period of the voice waveform based on the logical segmentation start and end time of the pre-labeled data. Based on the semi-transparent mask, the pre-labeled data and the logical segmentation start and end time corresponding to the pre-labeled data are precisely adjusted, thus realizing visual and precise segmentation and labeling.

[0054] (3) Rich annotation information: It can complete the text content annotation of the railway dispatching voice files to be annotated. This part of the annotation data can be used for the training and tuning of the dispatching voice transcription algorithm; it can complete the voiceprint clustering annotation of the railway dispatching voice files to be annotated. This part of the annotation data can be used for the training and tuning of the human voice clustering algorithm of multiple dispatchers in a single channel; it can complete the annotation of the ambient sound category, and this part of the annotation data can be used for the training and tuning of the ambient sound recognition deep learning algorithm; it can complete the annotation of the silent segment. This part of the data and the annotation of the ambient sound can restore the dispatching scene to a higher degree;

[0055] (4) In view of the characteristics of railway dispatching recordings, a series of small strategies are provided to improve the efficiency of data annotation and ensure effectiveness, such as the semi-transparent mask non-overlapping strategy and the preset integration strategy, to generate high-quality annotated data.

[0056] (5) In view of the characteristics of dispatching speech, a standardized annotation process was designed. Based on the principle of one person, one responsibility, the corpus was annotated first, then reviewed and finally sampled. The standardized process ensures the quality of the corpus at the institutional level. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is a flow chart of the railway dispatching voice annotation method of the present invention.

[0058] Figure 2 It is a schematic diagram of the pre-marking process of the present invention.

[0059] Figure 3 It is a schematic diagram of the process of adding a semi-transparent mask in the present invention.

[0060] Figure 4 It is a schematic diagram of a labeling task status list of the labeling system for railway dispatching speech of the present invention.

[0061] Figure 5 It is a work flow chart of the railway dispatching speech annotation system of the present invention. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0063] The present invention discloses a railway dispatching recording annotation system and method, which is dedicated to realizing accurate segmentation and annotation, and efficiently producing high-quality railway dispatching corpus with rich annotation information.

[0064] Example 1

[0065] To achieve the above purpose, this embodiment provides a method for marking railway dispatching speech, such as Figure 1 As shown, the following steps are included:

[0066] Acquire a railway dispatching voice file to be annotated, and perform pre-annotation processing on the railway dispatching voice file to be annotated based on an artificial intelligence voice annotation algorithm to obtain a pre-annotated file, wherein the pre-annotated file includes a plurality of pre-annotated data and a voice start and end time associated with each pre-annotated data;

[0067] The railway dispatching voice file to be annotated is displayed in the interface image in the form of a voice waveform; a plurality of pre-annotated data in the pre-annotated file are arranged and displayed in the interface image according to a preset sorting rule; a semi-transparent mask is added to the corresponding time period of the voice waveform based on the voice start and end time, and each semi-transparent mask is associated with a corresponding voice start and end time; and a preset mouse event is associated with each semi-transparent mask and each pre-annotated data; preferably, the preset mouse event is a double-click event;

[0068] Correct the pre-annotated file based on the semi-transparent mask and the preset correction strategy to obtain the annotated file:

[0069] When it is detected that the waveform start and end time corresponding to the semi-transparent mask changes, the waveform start and end time corresponding to the semi-transparent mask after the change is obtained, and the waveform start and end time is used to update and correct the speech start and end time associated with the semi-transparent mask;

[0070] When a preset mouse event is detected for the semi-transparent mask, the corresponding voice segment is searched for and played in the to-be-annotated railway dispatching voice file based on the voice start and end time associated with the semi-transparent mask, and after receiving manual input data, the pre-annotated data associated with the semi-transparent mask is updated with the manual input data;

[0071] or when detecting that the pre-annotated data has a preset mouse event, searching for a corresponding voice segment in the to-be-annotated railway dispatching voice file based on the voice start and end time associated with the pre-annotated data for playback, and after receiving manual input data, updating the pre-annotated data with the manual input data;

[0072] The annotated file and the railway dispatching voice file to be annotated are output as the annotation results.

[0073] The above-mentioned annotation method uses an artificial intelligence speech annotation algorithm to pre-annotate the railway dispatching voice file to be annotated to obtain a pre-annotated file; the pre-annotated file is manually corrected without the need for full manual annotation. The efficiency of railway dispatching corpus production is greatly improved through automatic pre-annotation and manual correction.

[0074] At the same time, the above-mentioned annotation method presents the railway dispatching voice file to be annotated in the form of a waveform, adds a semi-transparent mask to the corresponding time period of the voice waveform based on the logical segmentation start and end time of the pre-annotated data, and accurately adjusts the pre-annotated data and the logical segmentation start and end time corresponding to the pre-annotated data based on the semi-transparent mask, so that the annotator can visually adjust and correct the pre-annotated file sentence by sentence: on the one hand, it is judged whether the logical segmentation start and end time corresponding to each pre-annotated data is appropriate, and inappropriate logical segmentation is identified by observing the waveform covered by the semi-transparent mask and by playing the voice segment corresponding to the voice start and end time associated with the semi-transparent mask in the railway dispatching voice file to be annotated. For inappropriate logical segmentation, the waveform start and end time corresponding to the semi-transparent mask is adjusted to achieve fine-tuning of the logical segmentation, so as to achieve the purpose of not segmenting words or sentences; on the other hand, it is possible to judge whether the pre-annotated data is accurate, and adjust the pre-annotated data sentence by sentence according to the heard voice content.

[0075] In an exemplary embodiment, Figure 2As shown, the steps for obtaining the pre-marked file are as follows:

[0076] For the railway dispatching voice files to be annotated, a deep learning sound source separation model is used to separate the human voice signal and the ambient sound signal to obtain human voice signal segments and ambient sound signal segments;

[0077] For the vocal signal segments, the vocal signal segments are logically segmented using the vocal separation clustering model to obtain the vocal logical segments; the vocal logical segments are transcribed into text using the speech recognition model to obtain text annotation information; and the vocal logical segments are clustered based on the dispatcher identity, and the same virtual dispatcher identifier is annotated for multiple vocal logical segments belonging to the same dispatcher, and then the virtual dispatcher identifier is identified and associated with the dispatcher identity in the vocal voiceprint library using the voiceprint recognition model to obtain the dispatcher identity annotation information;

[0078] For the ambient sound signal fragments, the ambient sound separation clustering model is used to logically segment the ambient sound signal fragments to obtain ambient sound logical fragments; the ambient sound logical fragments are clustered based on the ambient sound categories, and multiple ambient sound logical fragments belonging to the same ambient sound are marked with the same virtual ambient sound identifier; the ambient sound category recognition model is used to identify and associate the virtual ambient sound identifier with the ambient sound identity in the ambient sound voiceprint library to obtain the ambient sound labeling information;

[0079] Based on a preset integration strategy, the text annotation information of the human voice logic segment, the dispatcher identity annotation information and the ambient sound category annotation information of the ambient sound logic segment are integrated to obtain a number of pre-annotated data, and finally a pre-annotated file containing the pre-annotated data and the voice start and end time of the pre-annotated data is output.

[0080] In railways, it is necessary to separate the caller from the callee in dispatching, and the conversations between two or more dispatchers are stored together in a single channel. The general voiceprint recognition model cannot label the identities of multiple dispatchers in the dispatching recording. By clustering the human voice logical fragments based on the dispatcher identity, and labeling the same virtual dispatcher identifier for multiple human voice logical fragments belonging to the same dispatcher, and then using the voiceprint recognition model, the virtual dispatcher identifier is identified and associated with the dispatcher identity in the human voiceprint library, and the dispatcher identity labeling information is obtained, so that logical segmentation can be performed according to the interaction order of the dispatchers in the later stage, thereby improving the labeling accuracy.

[0081] Since railway dispatching voice contains a lot of environmental sounds that are not dispatching dialogue content, such as whistle sounds, tunnel shuttle sounds, thunder, rain, etc., these environmental sounds do not belong to the dispatching content but can represent the situation at the time, such as whether it is passing through a tunnel, whether it is accompanied by rain, etc. This part of the data is of great value for highly restoring the dispatching scenario, so it is necessary to label the environmental sound segments; by using the environmental sound separation clustering model to logically segment the environmental sound signal fragments, obtain the environmental sound logical fragments; cluster the environmental sound logical fragments based on the environmental sound category, and label multiple environmental sound logical fragments belonging to the same environmental sound with the same virtual environmental sound identifier; use the environmental sound category recognition model to identify and associate the virtual environmental sound identifier with the environmental sound identity in the environmental sound voiceprint library to obtain the environmental sound labeling information, so that the later labeling personnel only need to check the pre-labeled environmental sound category, which greatly improves the labeling efficiency.

[0082] It can be understood that the environmental sound voiceprint library pre-stores identity information of common sounds such as whistle sounds, tunnel shuttle sounds, thunder, rain sounds, etc.

[0083] In view of the multi-terminology characteristic of dispatching speech, the present invention also provides a railway term library retrieval entrance in the process of adjusting the pre-annotated data sentence by sentence according to the heard speech content, so as to facilitate the annotators to check the railway term library in time when encountering uncertain dispatching content, thereby improving the annotation efficiency and ensuring the annotation quality.

[0084] Similarly, in the process of adjusting the pre-annotated data sentence by sentence according to the heard voice content, the present invention can also provide an environmental sound voiceprint library list. The annotator only needs to check the corresponding option list in the list to change the environmental sound category without manual filling, thereby improving the annotation efficiency.

[0085] Furthermore, the preset integration strategy executes:

[0086] Traversing the human voice logical segments, for each human voice logical segment, determining whether there is an ambient sound logical segment whose speech start time is greater than or equal to the speech start time of the human voice logical segment;

[0087] If so, it is further determined whether the voice ending time of the ambient sound logic segment is greater than or equal to the voice ending time of the human voice logic segment; if so, it is determined that the human voice logic segment is within the ambient sound logic segment, and the ambient sound logic segment is subjected to secondary logical segmentation according to the voice start and end time of the human voice logic segment; otherwise, it is determined that the human voice logic segment partially overlaps with the ambient sound logic segment, and the ambient sound logic segment is subjected to secondary logical segmentation according to the voice start time of the human voice logic segment;

[0088] If not, then further determine whether there is an ambient sound logic segment whose voice termination time is less than or equal to the voice termination time of the human voice logic segment. If so, determine that the ambient sound logic segment is within the human voice logic segment and do not perform any operation; otherwise, determine that the human voice logic segment partially overlaps with the ambient sound logic segment, and perform a secondary logical segmentation on the ambient sound logic segment according to the voice termination time of the human voice logic segment;

[0089] Traverse each voice start and end time to determine whether there are both human voice logic segments and ambient sound logic segments. If so, integrate the dispatcher identity annotation information, text annotation information, and ambient sound category annotation information corresponding to the current voice start and end time to form the pre-annotated data of the current voice start and end time; if not, use the dispatcher identity annotation information and text annotation information, or ambient sound category annotation information corresponding to the current voice start and end time as the pre-annotated data of the current voice start and end time;

[0090] Arrange the pre-annotated data of each speech start and end time in the order of the speech start and end time.

[0091] The above preset integration strategy can handle the situation where the human voice logic segment and the ambient sound logic segment overlap, and integrate the overlapping ambient sound logic segment into the annotation data of the human voice logic segment, so that the annotation personnel can intuitively feel the environment during scheduling.

[0092] As a method for labeling railway dispatching voice in an embodiment of the present invention, further, the present invention displays the voice file to be labeled horizontally in a waveform display area in the interface image in a waveform manner, and displays the pre-labeled data vertically in chronological order in the labeling content display area in the interface image.

[0093] It can be understood that the visualized waveform can vividly represent the volume of the voice file to be annotated. There is a waveform when there is sound, and there is no waveform in the silent segment, which is displayed as an interrupted blank. The larger the volume, the larger the waveform amplitude, and the smaller the volume, the smaller the waveform amplitude. If the voice volume is stable, the waveform tends to be a straight line, and if the voice volume fluctuates, the waveform tends to be a wavy line that floats up and down, which makes it easier to judge whether the logical segmentation corresponding to each pre-annotated data is reasonable based on the shape of the waveform (presence and size) in the later stage.

[0094] Furthermore, the timeline of the current railway dispatching voice file to be annotated is displayed at the bottom of the waveform display area, thereby providing a timeline reference for the annotators to accurately adjust the sentence segmentation points.

[0095] Furthermore, while displaying the waveform, the waveform display area also provides a playback control function, including overall playback, loop playback, previous sentence / next sentence playback, etc. For example, in a loud voice dispatching environment, it is inevitable that the annotator cannot hear clearly or cannot determine the voice content, and needs to play it repeatedly to determine it. The previous sentence / next sentence playback provided by the present invention can accurately play the specific sentence content, which is convenient for the annotator to quickly and repeatedly play a certain uncertain annotation content, without having to play it as a whole every time to the position that needs to be played repeatedly, greatly saving the annotation time cost.

[0096] It can be understood that the addition of the semi-transparent mask can help the annotator determine whether the logical segmentation of the speech segment to be annotated is reasonable; in specific implementation, according to the logical segmentation start time and the logical segmentation end time corresponding to the pre-annotated data, a semi-transparent mask is added to the corresponding time period of the speech waveform, and each semi-transparent mask is associated with one of the speech start and end times. Since the pre-annotated data is associated with the corresponding speech start and end times, the semi-transparent mask is also associated with the pre-annotated data, so that when the pre-annotated data is selected, the semi-transparent mask associated with the pre-annotated data is also in a selected state, and then the annotator can quickly and intuitively make a preliminary judgment on whether the logical segmentation of the speech segment to be annotated is appropriate through the shape of the waveform segment covered by the semi-transparent mask.

[0097] Furthermore, when it is determined that there are inappropriate segmentation points by observing the waveform covered by the semi-transparent mask and by playing the speech segment to be annotated corresponding to the semi-transparent mask area, the start and end times of the waveform corresponding to the semi-transparent mask are fine-tuned by manually dragging the semi-transparent mask to achieve the purpose of not segmenting words or sentences.

[0098] It should be noted that in the corpus annotation, the speech start and end times corresponding to different pre-annotated data cannot overlap, otherwise the pre-annotated data will be invalid and data resources will be wasted. Therefore, in the process of manually dragging the semi-transparent mask to correct the speech start and end times associated with the pre-annotated data, the following steps are performed:

[0099] When it is detected that the start and end time of the waveform corresponding to the semi-transparent mask has changed, that is, there is an overall dragging operation of the semi-transparent mask, there is a dragging operation of the left boundary line of the semi-transparent mask, and / or there is a dragging operation of the right boundary line of the semi-transparent mask, it is also necessary to determine whether the current operation is valid based on the semi-transparent mask non-overlapping strategy.

[0100] Specifically, the semi-transparent mask non-overlapping strategy also has three corresponding settings, as follows:

[0101] After the overall dragging operation of the semi-transparent mask is performed, it is determined whether the time of the left boundary line of the semi-transparent mask is greater than or equal to the time of the right boundary line of the previous semi-transparent mask. If so, it is further determined whether the time of the right boundary line of the semi-transparent mask is less than or equal to the time of the left boundary line of the next semi-transparent mask. If so, it is determined that the current operation is valid; otherwise, it is determined that the current operation is invalid;

[0102] After the dragging operation of the left border line of the semi-transparent mask is performed, it is determined whether the time of the left border line of the semi-transparent mask is greater than or equal to the time of the right border line of the previous semi-transparent mask. If so, it is determined that the current operation is valid, otherwise it is determined that the current operation is invalid;

[0103] After the drag operation of the right border line of the semi-transparent mask is performed, it is determined whether the time of the right border line of the semi-transparent mask is less than or equal to the time of the next left border line of the semi-transparent mask. If so, the current operation is determined to be valid, otherwise the current operation is determined to be invalid.

[0104] By executing the non-overlapping strategy of the semi-transparent masks, it can be ensured that the adjusted semi-transparent mask will not overlap with the semi-transparent masks before and after.

[0105] Furthermore, since the dispatch voice conversations are compact and contain a large number of short conversations such as "um, ah, okay, sure", the pre-annotation stage easily identifies the conversations of multiple dispatchers as one annotation content, and the annotators need to split the content of this type of situation based on what they hear; then, since this type of content is brief and has a short timeline, it is not easy for the annotators to determine the split points. For this reason, a waveform zoom function is also provided in the waveform display area. The annotators can zoom in and out the waveform by dragging the zoom button, which makes it easier for the annotators to accurately select the split points.

[0106] Furthermore, since the boundary lines of conventional semi-transparent masks are not easily captured by the mouse, it is not easy for the annotator to operate and it takes a certain amount of time to capture before the draggable icon can be triggered. For this reason, the present invention expands the mouse capture area on the left and right boundaries of the semi-transparent mask and adds a draggable mark. When the corresponding text sentence in the annotation content display area is clicked or the semi-transparent mask is clicked, the semi-transparent mask area is highlighted and the draggable mark is presented. The annotator only needs to place the mouse in the capture area without precise selection to drag the boundary line of the semi-transparent mask.

[0107] In the specific implementation, in order to facilitate the identification of the dispatcher's identity and the dispatcher's dispatch interaction sequence, the same color can be assigned to the semi-transparent masks corresponding to the pre-labeled data belonging to the same dispatcher according to the dispatcher's role, and the dispatcher's dispatch interaction sequence can be intuitively displayed through color differentiation.

[0108] Furthermore, when the identity of the dispatcher in the pre-annotated data changes, the color of the semi-transparent mask associated with the pre-annotated data also changes in conjunction. For example, the auxiliary pre-annotation marks the dispatch content originally belonging to dispatcher A as dispatcher B. After the dispatcher identity is visually adjusted, the color of the semi-transparent mask associated with the pre-annotated data is automatically adjusted from the color of dispatcher B to the color of dispatcher A.

[0109] In addition, the labeling personnel can customize the semi-transparent mask color. When one of the semi-transparent mask colors is modified, other pre-labeled data associated with the semi-transparent mask belongs to the pre-labeled data of the same dispatcher, and the semi-transparent mask colors associated with it also change in conjunction.

[0110] It is understandable that Figure 3 As shown, in the process of manually correcting the pre-annotated file based on the semi-transparent mask and the preset correction strategy, it is also possible to detect whether there is a semi-transparent mask adding operation on the speech waveform. If so, the waveform start and end time corresponding to the newly added semi-transparent mask is obtained as the newly added speech start and end time, the newly added speech start and end time is added to the pre-annotated file, and the newly added speech start and end time is associated and bound with the newly added semi-transparent mask, and the newly added speech start and end time is arranged and displayed in the interface image according to the preset sorting rule, and the preset mouse event is associated with the newly added semi-transparent mask;

[0111] It is further detected whether there is a preset mouse event for the newly added semi-transparent mask. If so, the corresponding voice segment is searched in the railway scheduling voice file to be annotated based on the voice start and end time associated with the newly added semi-transparent mask for playback. After receiving the manually input data, the manually input data is associated and bound with the newly added voice start and end time as the newly added pre-annotated data, and added to the pre-annotated file. The newly added pre-annotated data is displayed in the interface image, and the preset mouse event is associated with the newly added pre-annotated data.

[0112] In specific implementation, when the annotator wants to manually segment out a complete speech segment and add a new segment of annotation data, he or she can also manually drag the mouse in the speech waveform display area, and manually add the semi-transparent mask to the corresponding waveform segment of the speech waveform according to the speech start and end time of the logical segment to be segmented; at this time, the presence of a semi-transparent mask addition operation on the speech waveform will be detected, and the waveform start and end time corresponding to the newly added semi-transparent mask will be obtained as the newly added speech start and end time, the newly added speech start and end time will be added to the pre-annotation file, and the newly added speech start and end time will be associated and bound with the newly added semi-transparent mask, and the newly added semi-transparent mask will be associated with a preset mouse event; at the same time, the newly added speech start and end time will be arranged and displayed in the interface image according to the preset sorting rules;

[0113] After adding the semi-transparent mask, it is necessary to further manually annotate the voice segments corresponding to the newly added voice start and end time associated with the newly added semi-transparent mask; at this time, double-click the newly added semi-transparent mask to preset the mouse event, and search for the corresponding voice segment in the railway scheduling voice file to be annotated based on the voice start and end time associated with the newly added semi-transparent mask for playback, manually input the pre-annotated data according to the heard voice content, associate the pre-annotated data with the newly added voice start and end time as the newly added pre-annotated data, add it to the pre-annotated file, and associate the preset mouse event with the newly added pre-annotated data, and display the newly added pre-annotated data in the interface image, so that the pre-annotated data can be clicked later to perform other operations.

[0114] It should be noted that when the newly added annotation data is inserted into the interface image according to the scheduling interaction timeline, it is not inserted after the last pre-annotated data by default, but is inserted into the corresponding position according to the start and end time of the voice. This can have the effect of intuitively displaying the dispatcher's scheduling interaction order.

[0115] Furthermore, the newly added semi-transparent mask also needs to ensure that it does not overlap with the front and back semi-transparent masks; specifically, the semi-transparent mask non-overlapping strategy is:

[0116] After executing the translucent mask adding operation, determine whether the time of the left boundary line of the translucent mask is greater than or equal to the time of the right boundary line of the previous translucent mask. If so, further determine whether the time of the right boundary line of the translucent mask is less than or equal to the time of the left boundary line of the next translucent mask. If so, determine that the current operation is valid; otherwise, determine that the current operation is invalid.

[0117] Furthermore, the annotator can annotate the silent segment by manually adding pre-annotated data, and the specific text annotation content is [silent segment, silent remarks].

[0118] Silent annotation is of great value in highly restoring scheduling scenarios. In addition, when the voice file is played as a whole later, the system will automatically skip the silent segment and only play the sound part, shortening the playback time and improving annotation efficiency.

[0119] Example 2

[0120] This embodiment provides a railway dispatching speech annotation system, including:

[0121] A labeling task acquisition module, used to acquire a labeling task, wherein the labeling task includes a railway dispatching voice file to be labeled;

[0122] An algorithm-assisted pre-annotation module is used to perform pre-annotation processing on the railway dispatching voice file to be annotated based on an artificial intelligence voice annotation algorithm to obtain a pre-annotated file, wherein the pre-annotated file includes a plurality of pre-annotated data and the voice start and end time associated with each pre-annotated data;

[0123] The marking task display module is used to display the railway dispatching voice file to be marked in the interface image in the form of voice waveform; arrange and display several pre-marked data in the pre-marked file in the interface image according to the preset sorting rules; add a semi-transparent mask to the corresponding time period of the voice waveform based on the voice start and end time, and each semi-transparent mask is correspondingly associated with one of the voice start and end time; and associate a preset mouse event with each semi-transparent mask and each pre-marked data;

[0124] The visual correction module is used to correct the pre-annotated file based on the semi-transparent mask and the preset correction strategy to obtain the annotated file:

[0125] When it is detected that the waveform start and end time corresponding to the semi-transparent mask changes, the waveform start and end time corresponding to the semi-transparent mask after the change is obtained, and the waveform start and end time is used to update and correct the speech start and end time associated with the semi-transparent mask;

[0126] When a preset mouse event is detected for the semi-transparent mask, the corresponding voice segment is searched for and played in the to-be-annotated railway dispatching voice file based on the voice start and end time associated with the semi-transparent mask, and after receiving manual input data, the pre-annotated data associated with the semi-transparent mask is updated with the manual input data;

[0127] or when detecting that the pre-annotated data has a preset mouse event, searching for a corresponding voice segment in the to-be-annotated railway dispatching voice file based on the voice start and end time associated with the pre-annotated data for playback, and after receiving manual input data, updating the pre-annotated data with the manual input data;

[0128] The labeling task result output module is used to output the labeling file and the railway dispatching voice file to be labeled as the labeling result.

[0129] In a specific implementation, the algorithm-assisted pre-labeling module uses a deep learning sound source separation model to separate the human voice signal and the ambient sound signal for each voice segment to be labeled, so as to obtain a human voice signal segment and / or an ambient sound signal segment; transcribes each voice signal segment into text to obtain text labeling information; uses a deep learning voiceprint identity recognition model to process the human voice signal segment to obtain the dispatcher identity labeling information; uses a deep learning ambient sound recognition model to process the ambient sound signal segment to obtain the ambient sound category labeling information; and arranges the voice segments to be labeled in chronological order, integrates the text labeling information, the dispatcher identity labeling information and the ambient sound category labeling information of each voice segment to be labeled, obtains the pre-labeling data, and finally outputs a pre-labeling file containing the start time, end time, duration and pre-labeling data.

[0130] Furthermore, it also includes a voice signal preprocessing module, an audit module, a sampling module, a task management module and a role management module;

[0131] The role management module is used to set different roles and assign different processing permissions to different roles; the roles include task administrators, labelers, reviewers, and sampling personnel;

[0132] The speech signal preprocessing module is used to perform speech detection processing, environmental sound suppression processing and data enhancement processing on the speech signal of the railway dispatching speech file to be marked;

[0133] The task management module is used for the task administrator to import the railway dispatching voice files to be annotated and export the valid corpus;

[0134] The audit module is used for auditors to receive tasks to be audited and conduct visual audit work, and after the audit is passed, the tasks to be audited and the audit results are converted into tasks to be sampled for release, and when the audit fails, the audit opinions are returned, and the tasks to be audited are returned to the annotators for re-annotation, wherein the tasks to be audited are the annotated results;

[0135] The sampling inspection module is used for sampling inspection personnel to receive the tasks to be sampled, conduct visual review work, and mark the tasks to be sampled as qualified for sampling inspection after the review is passed, and the marking results generated by the tasks are marked as valid corpus.

[0136] In view of the characteristics of scheduling voice, the present invention utilizes role classification to realize dedicated person and dedicated responsibility, and ensure the quality of corpus; the task administrator has the right to issue labeling tasks and export valid corpus, the labeling personnel can only receive the tasks to be labeled and carry out labeling, and after labeling is completed, the labeling task becomes the task to be reviewed, and the reviewer can only receive the task to be reviewed and carry out review, and if the review is passed, the review task becomes the task to be sampled, and if the sampled inspection is qualified, the labeling result generated by the task is valid corpus.

[0137] It can be understood that the labeling task of the present invention runs through the entire labeling process, and the status of the labeling task is different at different labeling stages. Figure 4 As shown;

[0138] The task management module is used for the task administrator to import the railway dispatching voice files to be annotated and export the valid corpus;

[0139] The audit module is used for auditors to receive tasks to be audited for visual audit work, and after the audit is passed, the audit task is converted into a task to be sampled for release, and when the audit fails, the audit opinion is returned, and the task to be audited is returned to the annotator for re-annotation, wherein the task to be audited is the annotated result;

[0140] The sampling inspection module is used for sampling inspection personnel to receive the tasks to be sampled, conduct visual review work, and mark the tasks to be sampled as qualified for sampling inspection after the review is passed, and the marking results generated by the tasks are marked as valid corpus.

[0141] Furthermore, the annotation process of the railway dispatching speech annotation system is as follows: Figure 5 As shown, the following steps are included:

[0142] Step 1: Open the railway dispatching voice visualization annotation system;

[0143] Step 2: The task administrator fills in basic information such as the name of the annotation task, sets the subsequent sampling "percentage", and imports the railway dispatching voice file to be annotated. One annotation task can contain multiple files to be annotated;

[0144] Step 3: The annotator receives the annotation task and calls the voice signal preprocessing function to process the voice signal of the railway dispatching voice file to be annotated through voice detection, ambient sound suppression, and data enhancement methods to improve the clarity and intelligibility of the voice.

[0145] Step 4, calling the algorithm-assisted pre-annotation module to perform pre-annotation processing, and outputting a pre-annotation file containing the pre-annotated data and the speech start and end time of the pre-annotated data;

[0146] Step 5: Visually display the railway dispatching voice files to be annotated and the pre-annotated files;

[0147] Step 6: The annotator visually adjusts the pre-annotated file sentence by sentence, and supports adding text annotation information, adding dispatcher identity annotation information, adding ambient sound annotation, and adding silent segment annotation to achieve precise segmentation and annotation, and output the annotation results;

[0148] Step 7: The reviewer receives the marking task marked by the annotator and conducts visual review, mainly to review whether the annotator's sentence segmentation, ambient sound marking, silent marking, content marking, dispatcher name, etc. are appropriate. If the review passes, the marking task will enter the next stage and wait for the sampling inspection. If the review fails, the "Review Opinion" will be returned to the annotator for re-marking; the "Review Opinion" explains the reason for the failure of the review, and the annotator can re-mark according to the prompts of the review opinion.

[0149] It can be understood that each text sentence has an audit result, and each annotation file has an audit result. If a text sentence fails the audit, the annotation file will also fail the audit.

[0150] Step 8: The sampling personnel receive the marking tasks reviewed by the reviewers, and carry out random sampling work according to the sampling "percentage" set when the tender task is released; the sampling principle is the same as the review principle. The review work can guarantee the validity of the marking results to a certain extent, and the sampling work further verifies the validity of the marking results; if the sampling inspection passes, the marking result will be marked as valid marking data. If the review fails, the "review opinion" will be returned and sent back to the marking personnel for re-marking.

[0151] Step 9: The valid annotated data that passes the random inspection can be exported by the task administrator. The exported annotated data includes the railway dispatching voice file to be annotated and the corresponding annotation file. The annotation file contains the voice start and end time, duration and text annotation information, dispatcher identity annotation information, ambient sound annotation, silent annotation and other information.

[0152] Example 3

[0153] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the railway dispatching speech annotation method described in Example 1 when executing the program.

[0154] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, ordinary technicians in the field should understand that the specific implementation methods of the present invention can still be modified or some technical features can be replaced by equivalents without departing from the spirit of the technical solution of the present invention, which should be included in the scope of the technical solution for protection of the present invention.

Claims

1. A method for labeling railway dispatching speech, characterized in that: The following steps are involved: Acquire a railway dispatching voice file to be annotated, and perform pre-annotation processing on the railway dispatching voice file to be annotated based on an artificial intelligence voice annotation algorithm to obtain a pre-annotated file, wherein the pre-annotated file includes a plurality of pre-annotated data and a voice start and end time associated with each pre-annotated data; The railway dispatching voice file to be annotated is displayed in the interface image in the form of a voice waveform; a plurality of pre-annotated data in the pre-annotated file are arranged and displayed in the interface image according to a preset sorting rule; a semi-transparent mask is added to the corresponding time period of the voice waveform based on the voice start and end time, and each semi-transparent mask is associated with a corresponding voice start and end time; and associating a preset mouse event with each semi-transparent mask and each pre-labeled data; Correct the pre-annotated file based on the semi-transparent mask and the preset correction strategy to obtain the annotated file: When it is detected that the waveform start and end time corresponding to the semi-transparent mask changes, the waveform start and end time corresponding to the semi-transparent mask after the change is obtained, and the waveform start and end time is used to update and correct the speech start and end time associated with the semi-transparent mask; When a preset mouse event is detected for the semi-transparent mask, the corresponding voice segment is searched for and played in the railway dispatching voice file to be annotated based on the voice start and end time associated with the semi-transparent mask, and after receiving manual input data, the pre-annotated data associated with the semi-transparent mask is updated with the manual input data; or when detecting that the pre-annotated data has a preset mouse event, searching for a corresponding voice segment in the to-be-annotated railway dispatching voice file based on the voice start and end time associated with the pre-annotated data for playback, and after receiving manual input data, updating the pre-annotated data with the manual input data; The annotated file and the railway dispatching voice file to be annotated are output as the annotation results.

2. The railway dispatching speech annotation method according to claim 1, characterized in that: The conditions for the waveform start and end time corresponding to the semi-transparent mask to change are that there is an overall drag operation of the semi-transparent mask, there is a drag operation of the left boundary line of the semi-transparent mask, and / or there is a drag operation of the right boundary line of the semi-transparent mask.

3. The railway dispatching speech annotation method according to claim 2 is characterized in that: During the manual correction of the pre-annotated file based on the semi-transparent mask and the preset correction strategy, it is also possible to detect whether there is a semi-transparent mask addition operation on the speech waveform. If so, the waveform start and end time corresponding to the newly added semi-transparent mask is obtained as the newly added speech start and end time, the newly added speech start and end time is added to the pre-annotated file, and the newly added speech start and end time is associated and bound with the newly added semi-transparent mask, and the newly added speech start and end time is arranged and displayed in the interface image according to the preset sorting rules, and the preset mouse event is associated with the newly added semi-transparent mask; It is further detected whether there is a preset mouse event for the newly added semi-transparent mask. If so, the corresponding voice segment is searched in the railway scheduling voice file to be annotated based on the voice start and end time associated with the newly added semi-transparent mask for playback. After receiving the manually input data, the manually input data is associated and bound with the newly added voice start and end time as the newly added pre-annotated data, and added to the pre-annotated file. The newly added pre-annotated data is displayed in the interface image, and the preset mouse event is associated with the newly added pre-annotated data.

4. The railway dispatching speech annotation method according to claim 3 is characterized by: When there is a translucent mask adding operation, a translucent mask overall dragging operation, a translucent mask left boundary line dragging operation, or a translucent mask right boundary line dragging operation, it is also necessary to determine whether the current operation is valid based on the translucent mask non-overlapping strategy.

5. The railway dispatching speech annotation method according to claim 4 is characterized in that: The non-overlapping strategy of the semi-transparent mask is: After performing a translucent mask adding operation or performing a translucent mask overall dragging operation, determining whether the time of the left boundary line of the translucent mask is greater than or equal to the time of the right boundary line of the previous translucent mask, if so, further determining whether the time of the right boundary line of the translucent mask is less than or equal to the time of the left boundary line of the next translucent mask, if so, determining that the current operation is valid; otherwise, determining that the current operation is invalid; After the dragging operation of the left border line of the semi-transparent mask is performed, it is determined whether the time of the left border line of the semi-transparent mask is greater than or equal to the time of the right border line of the previous semi-transparent mask. If so, it is determined that the current operation is valid, otherwise it is determined that the current operation is invalid; After the drag operation of the right border line of the semi-transparent mask is performed, it is determined whether the time of the right border line of the semi-transparent mask is less than or equal to the time of the next left border line of the semi-transparent mask. If so, the current operation is determined to be valid, otherwise the current operation is determined to be invalid.

6. The railway dispatching speech annotation method according to claim 1, characterized in that: The steps for obtaining the pre-annotated file are as follows: For the railway dispatching voice files to be annotated, a deep learning sound source separation model is used to separate the human voice signal and the ambient sound signal to obtain human voice signal segments and ambient sound signal segments; For the human voice signal segments, the human voice signal segments are logically segmented using the human voice separation clustering model to obtain human voice logical segments; the human voice logical segments are transcribed into text using the speech recognition model to obtain text annotation information; and clustering the human voice logical segments based on the dispatcher identity, and labeling multiple human voice logical segments belonging to the same dispatcher with the same virtual dispatcher identifier, and then using the voiceprint recognition model to identify and associate the virtual dispatcher identifier with the dispatcher identity in the human voiceprint library to obtain the dispatcher identity labeling information; For the ambient sound signal fragments, the ambient sound signal fragments are logically segmented using the ambient sound separation clustering model to obtain ambient sound logical fragments; the ambient sound logical fragments are clustered based on the ambient sound categories, and multiple ambient sound logical fragments belonging to the same ambient sound are marked with the same virtual ambient sound identifier; Using the ambient sound category recognition model, the virtual ambient sound identifier is identified and associated with the ambient sound identity in the ambient sound voiceprint library to obtain the ambient sound labeling information; Based on a preset integration strategy, the text annotation information of the human voice logic segment, the dispatcher identity annotation information and the ambient sound category annotation information of the ambient sound logic segment are integrated to obtain a number of pre-annotated data, and finally a pre-annotated file containing the pre-annotated data and the voice start and end time of the pre-annotated data is output.

7. The railway dispatching speech annotation method according to claim 6, characterized in that: The preset integration strategy performs: Traversing the human voice logical segments, for each human voice logical segment, determining whether there is an ambient sound logical segment whose speech start time is greater than or equal to the speech start time of the human voice logical segment; If so, it is further determined whether the voice ending time of the ambient sound logic segment is greater than or equal to the voice ending time of the human voice logic segment; if so, it is determined that the human voice logic segment is within the ambient sound logic segment, and the ambient sound logic segment is subjected to secondary logical segmentation according to the voice start and end time of the human voice logic segment; otherwise, it is determined that the human voice logic segment partially overlaps with the ambient sound logic segment, and the ambient sound logic segment is subjected to secondary logical segmentation according to the voice start time of the human voice logic segment; If not, then further determine whether there is an ambient sound logic segment whose voice termination time is less than or equal to the voice termination time of the human voice logic segment. If so, determine that the ambient sound logic segment is within the human voice logic segment and do not perform any operation; otherwise, determine that the human voice logic segment partially overlaps with the ambient sound logic segment, and perform a secondary logical segmentation on the ambient sound logic segment according to the voice termination time of the human voice logic segment; Traverse each speech start and end time to determine whether there are both human voice logic segments and ambient sound logic segments. If so, integrate the dispatcher identity labeling information, text labeling information, and ambient sound category labeling information corresponding to the current speech start and end time to form pre-labeled data for the current speech start and end time. If it does not exist, the dispatcher identity labeling information and text labeling information, or the ambient sound category labeling information corresponding to the current speech start and end time is used as the pre-labeling data of the current speech start and end time; Arrange the pre-annotated data of each speech start and end time in the order of the speech start and end time.

8. A railway dispatching speech annotation system, characterized in that: include: A labeling task acquisition module, used to acquire a labeling task, wherein the labeling task includes a railway dispatching voice file to be labeled; An algorithm-assisted pre-annotation module is used to perform pre-annotation processing on the railway dispatching voice file to be annotated based on an artificial intelligence voice annotation algorithm to obtain a pre-annotated file, wherein the pre-annotated file includes a plurality of pre-annotated data and the voice start and end time associated with each pre-annotated data; The marking task display module is used to display the railway dispatching voice file to be marked in the interface image in the form of voice waveform; arrange and display several pre-marked data in the pre-marked file in the interface image according to the preset sorting rules; add a semi-transparent mask to the corresponding time period of the voice waveform based on the voice start and end time, and each semi-transparent mask is associated with a corresponding voice start and end time; and associating a preset mouse event with each semi-transparent mask and each pre-labeled data; The visual correction module is used to correct the pre-annotated file based on the semi-transparent mask and the preset correction strategy to obtain the annotated file: When it is detected that the waveform start and end time corresponding to the semi-transparent mask changes, the waveform start and end time corresponding to the semi-transparent mask after the change is obtained, and the waveform start and end time is used to update and correct the speech start and end time associated with the semi-transparent mask; When a preset mouse event is detected for the semi-transparent mask, the corresponding voice segment is searched for and played in the railway dispatching voice file to be annotated based on the voice start and end time associated with the semi-transparent mask, and after receiving manual input data, the pre-annotated data associated with the semi-transparent mask is updated with the manual input data; or when detecting that the pre-annotated data has a preset mouse event, searching for a corresponding voice segment in the to-be-annotated railway dispatching voice file based on the voice start and end time associated with the pre-annotated data for playback, and after receiving manual input data, updating the pre-annotated data with the manual input data; The labeling task result output module is used to output the labeling file and the railway dispatching voice file to be labeled as the labeling result.

9. The railway dispatching speech annotation system according to claim 8, characterized in that: It also includes a voice signal preprocessing module, an audit module, a sampling module, a task management module, and a role management module; The role management module is used to set different roles and assign different processing permissions to different roles; the roles include task administrators, labelers, reviewers, and sampling personnel; The speech signal preprocessing module is used to perform speech detection processing, environmental sound suppression processing and data enhancement processing on the speech signal of the railway dispatching speech file to be marked; The task management module is used for the task administrator to import the railway dispatching voice files to be annotated and export the valid corpus; The audit module is used for auditors to receive tasks to be audited and conduct visual audit work, and after the audit is passed, the tasks to be audited and the audit results are converted into tasks to be sampled for release, and when the audit fails, the audit opinions are returned, and the tasks to be audited are returned to the annotators for re-annotation, wherein the tasks to be audited are the annotated results; The sampling inspection module is used for sampling inspection personnel to receive the tasks to be sampled, conduct visual review work, and mark the tasks to be sampled as qualified for sampling inspection after the review is passed, and the marking results generated by the tasks are marked as valid corpus.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the railway dispatching speech annotation method as described in any one of claims 1 to 7 when executing the program.

Images