VTS service out-of-area report ship detection method fusing VHF communication voice information

Abstract

The invention relates to a VTS service out-of-area report ship detection method fused with VHF communication voice information. The method comprises the following steps: firstly, acquiring VHF communication voice data and AIS data, and preprocessing; training a VHF speech recognition model, and translating the VHF speech recognition model into corpus text data through the trained VHF speech recognition model; then extracting the corpus text data based on a CasRel entity relationship extraction model to obtain feature information; dividing the feature information into two types, including the feature information containing the current position feature information and the feature information not containing the current position feature information; and respectively carrying out position detection so as to obtain a VTS service area external report ship detection result. Compared with the prior art, the method has the advantages that VTS service area external report ship detection is achieved, the method is rapid and efficient, the method conforms to the maritime affair field, accurate detection is achieved, multi-dimension is converted into low dimension, the problem complexity is greatly reduced, and calculation is simplified.

Description

Technical Field

[0001] The invention relates to the technical field of ship detection, in particular to a VTS out-of-service-area reporting ship detection method integrating VHF communication voice information. Background Art

[0002] With the development of economic globalization, the flow of goods between countries is frequent. The transportation of most goods is currently mainly completed through water transportation. The development of water transportation is inseparable from the management of the ship traffic management center. The ship traffic management center continuously supervises and provides services to the ships in the waters under its jurisdiction through voice interaction and other methods. Traffic control personnel must always maintain a high degree of concentration during work so that they can receive information from ship VHF (Very High Frequency, VHF), ship automatic identification system (Automatic Identification System, AIS), video surveillance (Closed Circuit Television, CCTV) and other equipment at any time, and quickly analyze it and make timely decisions. At present, the number of ships is increasing, and there is a phenomenon that some ships have not yet arrived at the service area of ​​vessel traffic service (VTS) but have reported to traffic control personnel. Such invalid reporting information will not only occupy the limited human resources and reduce the efficiency of traffic control, but also increase the workload of traffic control personnel, who already have a complicated job.

[0003] At present, there is no good solution for detecting ships reporting outside the VTS service area. At present, the research is mainly based on AIS data, and uses data intelligence to analyze the ship's trajectory and predict its trajectory status to assist traffic control personnel in making decisions. A small number of studies are based on the text data of the communication corpus between ships and traffic control personnel, and use deep learning models to mine key information such as ship name, destination and berthing, so as to assist traffic control personnel in making decisions.

[0004] The above method cannot detect ships reporting outside the VTS service area, and it also cannot effectively free traffic control personnel from the complicated, repetitive and inefficient work of dealing with various VHF ship reports. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a method for detecting ships reporting outside the VTS service area by integrating VHF communication voice information. By further processing the key information in the mined VHF corpus, combining AIS data, and utilizing both VHF corpus and AIS system data, accurate detection of ships outside the VTS service area can be achieved. This method can expand the water intelligent transportation system, reduce the work pressure of traffic control, and improve the effectiveness of ship reporting information.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] According to one aspect of the present invention, a method for detecting ships reporting outside the VTS service area by integrating VHF communication voice information is provided. The method combines AIS data with VHF communication corpus text, and extracts corpus text data based on CasRel entity relations, comprising the following steps:

[0008] S1, obtain VHF communication voice data and AIS data, and perform pre-processing;

[0009] S2, dividing the pre-processed VHF communication voice data into a first data set and a second data set; training a VHF speech recognition model using the first data set, and translating the second data set into corpus text data using the trained VHF speech recognition model;

[0010] The VHF speech recognition model includes an acoustic model and a language model;

[0011] S3, extracting the corpus text data obtained in S2 based on the CasRel entity relationship extraction model to obtain feature information;

[0012] S4, dividing the characteristic information obtained in S3 into two categories, including: characteristic information including the current position and characteristic information not including the current position; and performing position detection respectively, thereby obtaining the detection result of the ship reporting outside the VTS service area. Further, the VHF speech recognition model includes an acoustic model and a language model.

[0013] As a preferred solution, the VHF communication voice data in S1 is collected by a VHF voice collection device, and the AIS data is collected by an AIS data collection device;

[0014] Further, the VHF voice collection device is composed of a VHF receiving antenna, a VHF receiver and a storage device, and is used to collect VHF communication voice data of different VHF channels;

[0015] Furthermore, the AIS data acquisition device is composed of an AIS receiving antenna, an AIS receiver and a storage device, and is used to receive and store AIS data within a certain range. As a preferred solution, the preprocessing in S1 includes data cleaning, removing incomplete sentences from the maritime text corpus and segmenting long sentences;

[0016] Furthermore, for VHF communication voice data: the waveform is segmented and sliced ​​according to the voiceprint recognition technology, so that each slice corresponds to a complete shipping conversation, and a time tag is added to the conversation;

[0017] Furthermore, for AIS data: the ship name in the AIS data is matched with the ship name library to correct the ship name errors or inconsistencies in the AIS data, while ensuring that other information is correct.

[0018] As a preferred solution, the specific steps of training the VHF speech recognition model in S2 include:

[0019] Training the acoustic model: from the acquired VHF communication voice data, divide the first data set and the second data set according to a preset ratio, extract features from the data in the first data set, use the extracted features to train the general acoustic model, evaluate the model performance, optimize the model parameters, and obtain an acoustic model suitable for the maritime field;

[0020] Training language model: Collect text information in the maritime field, build a maritime text database, use its text data to train a general language model, evaluate model performance, optimize model parameters, and obtain a language model that fits the maritime field.

[0021] Furthermore, the VHF speech recognition model training process includes:

[0022] Acquire VHF voice audio data and maritime field text data; divide the preprocessed VHF communication voice data into a first data set and a second data set; train a VHF voice recognition model using the first data set;

[0023] The first data set includes a training set, a test set and a validation set;

[0024] The training datasets are used for training the existing general acoustic model and language model respectively;

[0025] Parameters are adjusted according to the performance of the model on the validation set, and the test set is used to evaluate the model performance, and finally the training process of the VHF speech recognition model is completed.

[0026] As a preferred solution, the specific process of translation in S2 is:

[0027] S201, extracting features from the remaining VHF voice data in the acquired VHF communication voice data, inputting the features into the acoustic model, and then decoding the syllable sequence output by the acoustic model to convert it into a word sequence;

[0028] S202, determining possible word sequences through the language model, and using a search algorithm to select the one with the highest probability from all possible word sequences as the output of the final text;

[0029] S203, post-processing the above text output, and finally manually sampling the output text, and retaining its corresponding time information, and storing it in the VHF corpus text database, that is, finally translating and obtaining corpus text data;

[0030] Furthermore, the VHF speech recognition model translation process includes:

[0031] Get sliced ​​VHF communication voice data, where each slice corresponds to a complete round of shipping traffic conversation;

[0032] Convert speech signals into digital signals, then process and analyze these digital signals to extract their features;

[0033] The corresponding text is identified from the extracted features through the pre-trained VHF speech recognition model, completing the VHF speech recognition model translation process.

[0034] Furthermore, the post-processing includes error correction, punctuation and standardization of professional terms in the maritime field.

[0035] As a preferred solution, the CasRel entity relationship extraction model in S3 includes a sentence encoding part and a cascade decoding part;

[0036] Furthermore, the sentence encoding part obtains the feature vector representation of each character through the Embedding layer and the bidirectional Transformer layer of BERT, and these vectors are used as inputs of the subject recognition and subject-object relationship recognition modules;

[0037] Furthermore, the cascade decoding part extracts subject information in the sentence through a subject recognition module, and then associates the object entity corresponding to the relationship through a subject-object relationship recognition module, thereby realizing entity relationship extraction.

[0038] As a preferred solution, the steps of extracting the corpus text data obtained in S2 based on the CasRel entity relationship extraction model in S3 and obtaining feature information in combination with the ship name library are specifically as follows:

[0039] S301, based on the CasRel entity relationship extraction model, extract feature information from the corpus text data obtained in S2, the feature information including the subject, i.e., the ship name, the object, i.e., the place name, and the relationship;

[0040] S302, converting the ship name into corresponding pinyin or completing the ship name that is pinyin but lacks Arabic numerals in combination with the ship name database, so that it is consistent with the ship name structure in the AIS data;

[0041] S303, retaining its time information, according to the characteristics of the ship's reported position in the VHF corpus text;

[0042] There are relationships between ship names and place names, including: current location, future location, and ambiguous location;

[0043] Furthermore, the current location is the current location of the ship;

[0044] Furthermore, the future location is the location where the ship will go;

[0045] Furthermore, the ambiguous position refers to the ambiguity of the position information reported by the ship.

[0046] Furthermore, the CasRel entity relationship extraction model training process includes:

[0047] Get the text data containing VHF corpus;

[0048] Based on the data, the corpus data is annotated with ship names, locations, and relationship information on the corpus annotation platform and stored in JSON data format. In this process, the word units are first annotated based on sentences. After the annotation is completed, the data is divided into training set, validation set, and test set according to a specific ratio.

[0049] The obtained training set is used to train the CasRel entity relationship extraction model;

[0050] Parameters are adjusted according to the performance of the model on the validation set, and the test set is used to evaluate the model performance, and finally the training process of the CasRel entity relationship model is completed.

[0051] Furthermore, the specific ideas of the CasRel entity relationship extraction model include:

[0052] First, the preprocessed VHF corpus text data is input into the pre-trained Bert module. The core encoding part of this module is the Transformer structure, which adjusts the weight coefficient matrix to reflect the degree of association between characters in the same sentence, thereby outputting the word vector representation of each character.

[0053] After obtaining the word vector representation of each character, the word vector representation of each character is used to predict the start and end positions of the subject entity;

[0054] According to the predicted start and end positions of the subject entity, the vector representation of each subject entity is calculated by adding and averaging.

[0055] The vector representation of each character is combined with the vector representation of the predicted subject entity, and then a mapping function is built for each relationship by traversing the relationship dictionary. These functions will be used to predict the object entity.

[0056] The final output is the entity-relationship triple, which is the required feature information ('ship name', 'relationship', 'place name').

[0057] Furthermore, the relationship between the ship and the reported location is divided into three categories: current location, future location and fuzzy location. At the same time, in the above maritime corpus, there is also a situation where a ship name corresponds to multiple locations, that is, there are overlapping triples. At present, most methods for extracting triples are to extract subject entities and object entities. Each entity pair has its label, and the label is the relationship between the two entities for processing. However, this method is not applicable to the extraction of overlapping triples. The cascaded binary labeling framework can solve this problem well, that is, the CasRel entity relationship extraction model is used to extract the VHF corpus text. In order to facilitate the needs of subsequent calculations, the extracted feature information is divided into two categories: feature information containing the current location and feature information excluding the current location (future location, fuzzy location).

[0058] As a preferred solution, the characteristic information classification in S4 includes: including the current position characteristic information and not including the current position characteristic information;

[0059] Furthermore, the characteristic information not containing the current position includes: future position characteristic information and fuzzy position characteristic information.

[0060] As a preferred solution, the detection position in S4 is specifically:

[0061] Furthermore, for the characteristic information including the current location, the place name is obtained and searched in the VTS service area place name database, so as to detect the corresponding reporting ship outside the VTS service area;

[0062] Furthermore, for feature information that does not contain the current position, its ship name is obtained, and the ship name similarity is calculated between it and the ship name in the AIS data to obtain a similarity ranking with the ship name in the AIS data. Then, combined with a pre-set specific threshold, the ship name that meets the specific threshold and is most similar is screened out, and its ship point coordinates are obtained. The vector from the point to each vertex of the polygonal VTS service area is calculated and the angles between the vectors are summed to determine whether it is equal to 2π within the accuracy range. The ship point that is not equal to 2π is output, and the ship at this point corresponds to a ship reporting outside the VTS service area, thereby obtaining the detection result of a ship reporting outside the VTS service area.

[0063] As a preferred solution, the ship name similarity calculation specifically includes:

[0064] Calculate the overlapped character offset and non-overlapped character offset between the ship name string in the VHF corpus without the feature information of the current position and the ship name string in the AIS data;

[0065] Calculate the total offset between the two ship name strings, and then get the offset similarity, that is, the ship name similarity.

[0066] Compared with the prior art, the present invention has the following advantages:

[0067] 1. The method described in the present invention combines AIS data with VHF communication corpus text, and extracts corpus text data based on CasRel entity relationship, by obtaining VHF communication voice data and AIS data, and performing preprocessing; training the VHF speech recognition model, translating the obtained VHF communication voice data into corpus text data through the trained VHF speech recognition model; extracting the obtained corpus text data based on the CasRel entity relationship extraction model, and combining it with the ship name library to obtain feature information; dividing the obtained feature information into two categories, and performing position detection, thereby completing the detection of ships reporting outside the VTS service area. A solution is provided for detecting ships outside the VTS service area, and the detection of ships reporting outside the VTS service area is realized.

[0068] 2. The method described in the present invention is not only limited to the use of AIS information, but also takes into account the characteristics of the ship's reported position in the VHF corpus text itself. According to the position characteristics, the extracted feature information is divided into feature information containing the current position and feature information not containing the current position; through classification, the characteristics of each type of feature information can be used as much as possible to improve the detection accuracy. For the feature information containing the current position, its place name information is obtained and can be directly searched in the place name database, so as to realize fast and effective detection of reporting ships outside the VTS service area.

[0069] 3. The method described in the present invention obtains the ship name for the ship that does not contain the current position feature information, calculates the ship name similarity between it and the ship name in the AIS data, and obtains the similarity ranking of the ship name in the AIS data; and sets a specific threshold, takes the ship name that meets the specific threshold and is the most similar, obtains the coordinates of its ship point, calculates the vector from the point to each vertex of the polygonal VTS service area and sums the angle between the vectors, determines whether it is equal to 2π within the accuracy range, and outputs the ship point that is not equal to 2π, that is, the ship corresponding to the point is a reporting ship outside the VTS service area, thereby detecting the reporting ship outside the VTS service area, and when determining the location of similar ships, makes full use of AIS information, obtains the coordinate points of similar ships and the coordinates of each vertex of the polygonal VTS service area, and converts it into the problem of whether a point on a plane is within the polygon, from multi-dimensional to low-dimensional, greatly reducing the complexity of the problem and simplifying the calculation.

[0070] 4. The method of the present invention jointly judges the ship position information and ship name information in the VHF corpus text with the AIS data, trains the acoustic model and the language model, evaluates the model performance, optimizes the model parameters, and obtains an acoustic model that fits the maritime field; and realizes efficient and accurate detection of ships reporting outside the VTS service area; BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 A schematic diagram of the steps of a method for detecting ships outside a VTS service area that incorporates VHF communication voice information in the present invention;

[0072] Figure 2 This is a schematic diagram of the overall water area in an embodiment of the present invention;

[0073] Figure 3 It is a schematic diagram of the process architecture of a VTS out-of-service-area reporting ship detection method integrating VHF communication voice information in an embodiment of the present invention;

[0074] Figure 4 It is a schematic diagram of the process of training and translating a VHF speech recognition model in an embodiment of the present invention;

[0075] Figure 5 is a schematic diagram of a process for extracting feature information in an embodiment of the present invention;

[0076] Figure 6 is a diagram of the CasRel entity relationship model architecture in an embodiment of the present invention;

[0077] Figure 7 It is a schematic diagram of the process of detecting a ship including a current position characteristic information report in an embodiment of the present invention;

[0078] Figure 8The figure is a schematic diagram of the process of detecting a ship without reporting the current position characteristic information in an embodiment of the present invention. DETAILED DESCRIPTION

[0079] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0080] Example

[0081] In this embodiment, a VTS service area reporting ship detection method integrating VHF communication voice information is applied, and the method steps are as follows: Figure 1 As shown, the method applied comprises the following steps:

[0082] S1, obtain VHF communication voice data and AIS data, and perform pre-processing;

[0083] S2, dividing the pre-processed VHF communication voice data into a first data set and a second data set; training a VHF speech recognition model using the first data set, and translating the second data set into corpus text data using the trained VHF speech recognition model;

[0084] The VHF speech recognition model includes an acoustic model and a language model;

[0085] S3, extracting feature information from the corpus text data obtained in S2 based on the CasRel entity relationship extraction model;

[0086] S4. Divide the characteristic information obtained in S3 into two categories, including: characteristic information containing the current position and characteristic information not containing the current position; and perform position detection on each of them, so as to obtain detection results of ships reporting outside the VTS service area.

[0087] In this embodiment, the overall water area diagram is as follows: Figure 2 As shown, the water area mainly includes:

[0088] The scope of the VTS service area is the waters under the jurisdiction of the Vessel Traffic Service Center, and the shape of the waters is an irregular polygon (the edges of the polygonal waters will not intersect);

[0089] The overall water area is the water area studied in this study. The outer edge distance is the maximum distance that VTS can receive shipborne VHF voice communication, which is about 25 nautical miles. In order to facilitate subsequent calculations, the shape of the water area is proposed as a circle with VTS as the center and a radius of 25 nautical miles.

[0090] The area outside the VTS service area refers to the entire water area excluding the VTS service area.

[0091] In this embodiment, a flow chart of a method for detecting ships outside the VTS service area that integrates VHF communication voice information is shown in FIG. Figure 3 As shown;

[0092] In this embodiment, when step S1 is specifically implemented, data collection and processing are first performed to obtain VHF communication voice and AIS data. The data is acquired through a VHF communication voice acquisition device and an AIS data acquisition device. The VHF voice acquisition device includes a VHF receiving antenna, a VHF receiver and a VHF data storage device. The AIS data real-time acquisition device includes an AIS receiving antenna, an AIS receiver and an AIS data storage device. The acquired VHF communication voice data is segmented and sliced ​​according to the waveform using voiceprint recognition technology to ensure that each slice corresponds to a complete conversation. Secondly, a time tag is added to this conversation, and the processed data is saved in a VHF voice slice database. The ship name in the obtained AIS data is matched with the ship name library to correct possible ship name errors or inconsistencies in the AIS data. At the same time, other information is ensured to be correct, and the corrected data is stored in the corresponding AIS database.

[0093] In this embodiment, when the step S2 is specifically implemented, the language recognition and translation of the VHF data is performed, the features of the acquired VHF voice data are extracted and input into the pre-trained VHF voice recognition model to obtain the corpus text data, which is then post-processed. The translated corpus text is manually sampled based on the VHF communication voice data to ensure that the translation is correct and retain its time information. Finally, the corpus text data is saved in the corresponding VHF corpus text database.

[0094] In this embodiment, when implementing step S3, feature information is extracted, the VHF corpus text obtained in the above step S2 is preprocessed and input into the pre-trained CasRel entity relationship extraction model, and its corresponding time information is retained, the extracted information is combined with the ship name library, and the ship name information is converted into pinyin or completed.

[0095] In this embodiment, the ship reporting position in the maritime corpus text includes:

[0096] In this embodiment, the current maritime corpus text contains corpus such as: "Hello, center LU HUA 1, I have applied for anchoring at anchorage No. 7 and am reporting to you", in which the ship name is "LU HUA 1", the current position is "anchorage No. 7", and its future position is "two periods outside", and there are also corpus such as: "Hello, center HONG TAI 13, 1011 exit, um, I have applied to you to anchor at anchorage No. 4 and wait for the plan", in this corpus, it cannot be determined that the current position of the ship "HONG TAI 13" is "1011", and this case is that the ship position is ambiguous.

[0097] In this embodiment, considering this feature in the maritime corpus text, the relationship between the ship and the reported location is divided into three categories: current location, future location and fuzzy location. At the same time, in the above maritime corpus, there is also a situation where one ship name corresponds to multiple locations, that is, there are overlapping triples.

[0098] Currently, most methods for extracting triplets are to extract subject entities and object entities. Each entity pair has its label, and the label is processed based on the relationship between the two entities. However, this method is not applicable to extracting overlapping triplets. The cascaded binary labeling framework can solve this problem well.

[0099] In this embodiment, the CasRel entity relationship extraction model is used to extract the VHF corpus text. To facilitate the needs of subsequent calculations, the extracted feature information is divided into two categories: feature information containing the current position and feature information not containing the current position (future position, fuzzy position).

[0100] The acquired feature information is saved in the VHF corpus text feature information database.

[0101] In this embodiment, the feature information extracted from the obtained VHF corpus text data includes: subject: ship name, object: place name, and relationship.

[0102] In this embodiment, when step S4 is specifically implemented, the location information is first searched and retrieved. The name of the current location of the ship is obtained through the feature information containing the current location. The place name is searched in the place name database within the VTS service area, and the place names that are not in the place name database are output. In combination with their corresponding feature information, ships reporting outside the VTS service area are detected. For feature information that does not contain the current location, the ship name is obtained, and the ship name similarity is calculated with the ship name in the AIS data. A threshold is set to obtain the ship name in the AIS data that meets the threshold and has the greatest similarity. The AIS ship matched by the feature information is obtained, and its coordinates are obtained. The coordinates are compared with the coordinates of each vertex of the polygonal VTS service area, and the angle method is used to determine whether the ship point is within the polygonal VTS service area. The ship name that does not meet the judgment is output, and the ship reporting outside the VTS service area can be detected in combination with its ship name information.

[0103] In this embodiment, in step S2, the VHF speech recognition model is trained and translated. The process is as follows Figure 4 As shown in FIG. 1 , the VHF speech recognition model training and translation process specifically includes:

[0104] First, acoustic feature extraction is performed. VHF voice audio data and maritime text data are obtained. A certain proportion of data sets are separated from the acquired VHF communication voice data, that is, the corresponding pre-processed training set, test set and validation set are obtained through acoustic feature extraction and text data structure processing, and feature extraction is performed on them. The extracted features are used to train the general acoustic model, evaluate the model performance, optimize the model parameters, and obtain an acoustic model that is more suitable for the maritime field;

[0105] Secondly, collect text information in the maritime field, build a maritime text database, preprocess its text data, use the preprocessed data for training a general language model, evaluate the model performance, optimize the model parameters, adjust the parameters according to the performance of the model on the validation set, and use the test set to evaluate the model performance. Finally, complete the training process of the VHF speech recognition model and obtain a language model that is more suitable for the maritime field.

[0106] Then, features are extracted from the remaining VHF speech data and input into the above acoustic model. The syllable sequence output by the acoustic model is decoded and converted into a word sequence. The possible word sequences are determined by the above language model. The search algorithm is used to select the one with the highest probability from all possible word sequences as the final output text.

[0107] Finally, data correction is performed, including post-processing the text output of the final output text and manual sampling of the output text to ensure the accuracy of the VHF corpus text data, retain its corresponding time information, and store it in the VHF corpus text database.

[0108] In this embodiment, post-processing includes error correction, punctuation, and standardization of professional terms in the maritime field.

[0109] In this embodiment, feature information extraction is performed in step S3. The flowchart of feature information extraction is as follows: Figure 5 As shown, the feature information extraction process includes:

[0110] First, corpus text data is obtained from the VHF corpus text database. In this embodiment, the example corpus is: "Jianghai Guoyun is berthing at Jungong Road Wharf, reporting to you."

[0111] Perform corresponding processing on the corpus text data, input the processed data into the pre-trained CasRel entity relation extraction model to complete the extraction of feature information [('Jiang Haiguoyun', 'Current Location', '101')], retain the time information, and combine with the ship name library to convert the ship name in the feature information into the corresponding pinyin or complete it.

[0112] In this embodiment, that is, the ship name in the example ("Jianghai Guoyun") is translated into "JIANG HAI GUOYUN" through the ship name library to make the ship name structure consistent with AIS, which is convenient for subsequent calculations.

[0113] In this embodiment, the training process of the CasRel entity relation extraction model in S3 specifically includes:

[0114] First, manually summarize the location relation categories in the VHF corpus text data. For the preprocessed data, annotate the ship name, location, and relation information of the corpus data on the corpus annotation platform and store it in the JSON data format.

[0115] In this embodiment, the example text is: "Jianghai Guoyun is berthed at the Jungong Road Wharf and reports to you."

[0116] In this embodiment, its JSON data format should be "spo_list": [{"predicate": "Current Location", "object_type": {"@value": "Location"}, "subject_type": "vessel", "object": {"@value": "Jungong Road Wharf"}, "subject": "Jianghai Guoyun"}].

[0117] In this embodiment, after the annotation is completed, divide the training set, validation set, and test set according to a certain ratio, input the annotated data into the CasRel entity relation extraction model, use the test set to evaluate the model performance, and optimize according to the performance of the model on the test set to finally obtain a trained model.

[0118] Among them, for the CasRel entity relation extraction model, the general specific process of extracting entity relation triples includes:

[0119] Input the VHF corpus text data into the BERT module to obtain the vector representation of each character. The BERT module adjusts the weight coefficient matrix to reflect the correlation degree between characters in the VHF corpus text data, thereby outputting the word vector representation of each character. The following formula is used to calculate the correlation degree between characters in the VHF corpus text data:

[0120]

[0121] Among them, Attention represents the self-attention mechanism function; Softmax is the activation function; Q, K, and V are word vector matrices respectively; d k is the nesting dimension; T is the transpose symbol of the matrix; X i is the vector representation of each character; h N is the vector representation of the entire sentence.

[0122] In this embodiment, the start and end positions of the subject entity are predicted based on the vector representation of each character. The specific calculation method is as follows:

[0123]

[0124]

[0125] in, The start position label of the subject entity predicted for the i-th character; The end position tag of the subject entity predicted for the i-th character; X i is the vector representation of each character; that is, X i =h N [i]; W start is the weight matrix for predicting the starting position of the subject entity; W end is the weight matrix for predicting the end position of the subject entity; b start is the deviation of predicting the starting position of the subject entity; b end is the deviation of predicting the end position of the subject entity; σ is the fully connected layer.

[0126] In this embodiment, in the VHF corpus scenario, the subject entity is the ship name.

[0127] In this embodiment, based on the predicted start and end positions of the subject entity, the vector representation of each subject entity is obtained by adding and averaging, and the calculation is as follows:

[0128] V sub =average(X a , X b );

[0129] Among them, V sub is the obtained subject entity; average is the addition and averaging operation; X a The vector representation when the starting character position of the subject entity is the ath character; X b The vector representation when the ending character position of the subject entity is the bth character.

[0130] In this embodiment, the vector representation of each character is fused with the final information representation of the subject entity, and then a mapping function is created for each relationship. Next, the object entity is predicted based on the mapping function of each relationship, thereby completing the extraction of entity relationship triples. The specific formula is as follows:

[0131]

[0132]

[0133] in, The start position label of the object entity predicted for the i-th character, The end position tag of the object entity predicted for the i-th character; is the weight matrix for predicting the starting position of the object entity under relational mapping; X i is the vector representation of each character; that is, X i =h N [i]; V sub is the subject entity; The deviation of the starting position of the predicted object entity under relational mapping; is the weight matrix for predicting the end position of the object entity under relational mapping; It is the deviation of the predicted end position of the object entity under relational mapping.

[0134] In this embodiment, in the VHF corpus scenario, the object entity is a place name.

[0135] In this embodiment, the CasRel entity relationship model architecture is as follows: Figure 6 As shown in the figure, the model consists of two parts. The first part is the sentence encoding part: the feature vector representation of each character is obtained through the Embedding layer and the bidirectional Transformer layer of BERT. These feature vectors will be used as the input of the subject recognition and subject-object relationship recognition modules. The second part is the cascade decoding part: first, the subject information in the sentence is extracted through the subject recognition module, and then the object entity corresponding to the relationship is associated through the subject-object relationship recognition module.

[0136] In this embodiment, the ship detection process containing the current position feature information report is as follows: Figure 7 As shown, the specific process of reporting ship detection including current position characteristic information is as follows:

[0137] First, obtain the feature information containing the current position from the VHF corpus text feature information database;

[0138] Then, the corresponding ship location name information is obtained according to the obtained characteristic information, and a list containing multiple place names is generated; then the place name list created above is traversed, and a query is performed on each place name in the list in the VTS service area place name database;

[0139] In this embodiment, the place names with empty query results are finally added to the list of place names not in the place name database, and the corresponding ship information is obtained through the place name information in combination with the feature information. The obtained ship information is the ship reporting outside the VTS service area, and the detection is completed.

[0140] In this embodiment, the ship detection process without the current position feature information report is as follows: Figure 8 As shown, the specific process of reporting ship detection without current location feature information is as follows:

[0141] The AIS ship name information of a fixed time and space range and the VHF corpus text without the current position feature information are obtained from the AIS database and the VHF corpus text feature information database respectively, and the ship name information is obtained; then, the similarity between the two ship names is calculated by considering the overlapped character offset and the non-overlapped character offset between the two ship name strings, where the overlapped character offset is specifically calculated as follows:

[0142] O i =b i -a i ;

[0143] O=(O1,O2,O3,...,O n );

[0144] Dist(O)=|O1|+|O2|+|O3|+...+|O n |;

[0145] Among them, a i is the i-th character from left to right of the ship name string A, b i is the i-th character from left to right in the ship name string B, O i is the degree of difference in the positions of the overlapping characters that constitute the match, O is the offset vector composed of the offsets of all overlapping strings, and Dist(O) is the distance vector of all O i The sum of the absolute values ​​of , that is, the total offset of the overlapping characters.

[0146] In this embodiment, the smaller the total offset Dist(O) value of the overlapping characters is, the smaller the difference between the two ship name character strings A and B is, and the higher the similarity is;

[0147] In this embodiment, when considering non-overlapping characters, i.e., irrelevant characters, the offset can be regarded as the distance describing the similarity of the two ship name texts. Considering that the influence of non-overlapping characters on the similarity of the two ship name texts should be greater than that of overlapping characters, the absolute value of the offset of non-overlapping characters should be greater, and its calculation formula can be expressed as:

[0148]

[0149] Among them, O w For each non-overlapping character, len(A) is the length of the ship name string A, and len(B) is the length of the ship name string B. This ensures that the offset between two non-overlapping characters (i, j) is 2O. w Always greater than the absolute value of the offset between two overlapping characters (i,i) |a i -b i |.

[0150] In this embodiment, assuming that there are W non-overlapping characters between the ship name string A and the ship name string B, the sum of the offsets caused by all non-overlapping characters is recorded as the total offset of non-overlapping characters Dist(w), which is expressed as follows:

[0151]

[0152] Among them, the total offset Dist(A, B) is the sum of the total offset Dist(O) of overlapping characters and the total offset Dist(w) of non-overlapping characters, W is the number of non-overlapping characters. When the ship name string A and the ship name string B have no overlapping characters, W = len(A) + len(B). At this time, the difference between the two ship name strings is the largest, and the total offset also reaches the maximum value. At this time, Dist max The (A,B) formula is:

[0153]

[0154] Then, the offset similarity of the two ship name strings A and B can be obtained, and the calculation formula is:

[0155]

[0156] Among them, Sim(A,B) is the similarity of the ship name, that is, the offset similarity between the two ship name strings A and B.

[0157] In this embodiment, it can be seen from the above formula that when the ship name string A and the ship name string B are exactly the same, Dist(A, B) = 0, Sim(A, B) = 1; when the ship name string A and the ship name string B are completely different, that is, there are no overlapping characters, Dist(A, B) = Dist max (A,B),Sim(A,B)=0.

[0158] In this embodiment, if the ship name string A = "JIANG HAI GUO YUN", the ship name string B = "JIANGHAI", its Dist (O) = 0, Dist (w) = 8 × (17 + 9) / 2 = 104, Dist (A, B) = 104, Dist max (A,B)=338, Sim(A,B)=1-104 / 338=0.6923.

[0159] In this embodiment, the overlapping characters are matched as the nearest match, and the unmatched characters are regarded as non-overlapping characters, and the similarity is rounded to four decimal places;

[0160] In this embodiment, a certain similarity threshold is set as 0.8500, and the AI ​​S ship name that meets the threshold and has the greatest similarity is obtained, and its coordinates are obtained. The VHF corpus text ship name information that does not meet the threshold is discarded, and the vectors from the obtained ship coordinates to the coordinates of each vertex of the polygonal VTS service area water area are calculated, and the angles between the vectors are summed. The specific calculation formula is as follows:

[0161] θ i =arctan2(dx,dy);

[0162]

[0163] Among them, θ i It is expressed as the i-th vector angle, dx and dy represent the difference between two vectors in the x and y directions respectively, and θ represents the sum of all vector angles.

[0164] In this embodiment, a threshold of 0.01 radians is set. When θ is within the range of 2π±0.01, it is considered that θ is close to 2π, that is, the coordinate point of the ship is in the VTS service area. Otherwise, it is outside the VTS service area and the coordinates of the ship that do not meet the range are output.

[0165] In this embodiment, the specific ideas for detecting the ship position include:

[0166] According to the classification of characteristic information, first obtain the characteristic information containing the current location, extract the place name, search it in the VTS service area place name database, output the place name that is not in the place name database, and obtain the corresponding ship name information according to the characteristic information, that is, the detected ship reporting outside the VTS service area.

[0167] Secondly, obtain feature information without the current position, extract the ship name, and calculate the ship name similarity with the ship name in the AIS data. The ship name similarity calculation takes into account the overlapping offset and non-overlapping offset between the two ship name strings. The total offset is obtained by summing up and then the ship name similarity is obtained. A threshold is set, and the ship name that meets the threshold and has the largest similarity is taken as the matched AIS ship, and the location of the AIS ship is determined. When determining the location of the ship, the ship is regarded as a ship point and converted into a two-dimensional problem, that is, whether the point is in the polygonal VTS service area, and the ship point that is not in the VTS service area is output to obtain the corresponding ship name information, and finally complete the detection.

[0168] In this embodiment, the corresponding ship information is obtained according to the ship coordinates obtained above, and the obtained ship information is the reporting ship outside the VTS service area, and the detection is completed.

[0169] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for detecting ships reporting outside the VTS service area by integrating VHF communication voice information, characterized in that: The method combines AIS data with VHF communication corpus text and extracts corpus text data based on CasRel entity relations, including the following steps: S1, obtain VHF communication voice data and AIS data, and perform pre-processing; S2, dividing the pre-processed VHF communication voice data into a first data set and a second data set; training a VHF speech recognition model using the first data set, and translating the second data set into corpus text data using the trained VHF speech recognition model; The VHF speech recognition model includes an acoustic model and a language model; S3, extracting the corpus text data obtained in S2 based on the CasRel entity relationship extraction model to obtain feature information; S4, dividing the feature information obtained in S3 into two categories, including: feature information including the current position and feature information not including the current position; And carry out position detection respectively, so as to obtain the detection results of ships reporting outside the VTS service area.

2. A method for detecting ships outside the VTS service area that integrates VHF communication voice information according to claim 1, characterized in that: The VHF communication voice data in S1 is collected by a VHF voice collection device, and the AIS data is collected by an AIS data collection device; The VHF voice collection device is composed of a VHF receiving antenna, a VHF receiver and a storage device, and is used to collect VHF communication voice data of different VHF channels; The AIS data collection device is composed of an AIS receiving antenna, an AIS receiver and a storage device, and is used to receive and store AIS data within a certain range.

3. A VTS out-of-service-area reporting ship detection method integrating VHF communication voice information according to claim 1, characterized in that: The preprocessing in S1 includes data cleaning, removing incomplete sentences from the maritime text corpus and segmenting long sentences; For VHF communication voice data: segment and slice the waveform based on voiceprint recognition technology, so that each slice corresponds to a complete shipping conversation, and a time tag is added to the conversation; For AIS data: match the ship name in the AIS data with the ship name database, correct the ship name errors or inconsistencies in the AIS data, and ensure that other information is correct.

4. A method for detecting ships outside the VTS service area that integrates VHF communication voice information according to claim 1, characterized in that: The specific steps of training the VHF speech recognition model in S2 include: Training the acoustic model: from the acquired VHF communication voice data, divide the first data set and the second data set according to a preset ratio, extract features from the data in the first data set, use the extracted features to train the general acoustic model, evaluate the model performance, optimize the model parameters, and obtain an acoustic model suitable for the maritime field; Training language model: Collect text information in the maritime field, build a maritime text database, use its text data to train a general language model, evaluate model performance, optimize model parameters, and obtain a language model that fits the maritime field.

5. A method for detecting ships outside the VTS service area that integrates VHF communication voice information according to claim 1, characterized in that: The specific process of translation in S2 is: S201, extracting features from the remaining VHF voice data in the acquired VHF communication voice data, inputting the features into the acoustic model, and then decoding the syllable sequence output by the acoustic model to convert it into a word sequence; S202, determining possible word sequences through the language model, and using a search algorithm to select the one with the highest probability from all possible word sequences as the output of the final text; S203, post-processing the above text output, and finally manually sampling the output text, and retaining its corresponding time information, and storing it in the VHF corpus text database, that is, finally translating and obtaining corpus text data; The post-processing includes error correction, punctuation and standardization of professional terms in the maritime field.

6. A method for detecting ships reporting outside the VTS service area integrating VHF communication voice information according to claim 1, characterized in that: The CasRel entity relationship extraction model in S3 includes a sentence encoding part and a cascade decoding part; The sentence encoding part obtains the feature vector representation of each character through the Embedding layer and the bidirectional Transformer layer of BERT, and these vectors are used as inputs of the subject recognition and subject-object relationship recognition modules; The cascade decoding part extracts the subject information in the sentence through the subject recognition module, and then associates the object entity corresponding to the relationship through the subject-object relationship recognition module, thereby realizing entity relationship extraction.

7. A method for detecting ships outside the VTS service area that integrates VHF communication voice information according to claim 1, characterized in that: The steps described in S3 of extracting the corpus text data obtained in S2 based on the CasRel entity relationship extraction model and obtaining feature information in combination with the ship name library are as follows: S301, based on the CasRel entity relationship extraction model, extract feature information from the corpus text data obtained in S2, where the feature information includes the subject, i.e., the ship name, the object, i.e., the place name, and the relationship; S302, converting the ship name into corresponding pinyin or completing the ship name that is pinyin but lacks Arabic numerals in combination with the ship name database, so that it is consistent with the ship name structure in the AIS data; S303, retaining its time information, according to the characteristics of the ship's reported position in the VHF corpus text; There are relationships between ship names and place names, including current location, future location and ambiguous location; The current position is the current location of the vessel; The future position is the location where the vessel will go; The ambiguous position refers to the ambiguity of the position information reported by the ship.

8. A method for detecting ships outside the VTS service area that integrates VHF communication voice information according to claim 1, characterized in that: The characteristic information not including the current position in S4 includes: future position characteristic information and fuzzy position characteristic information.

9. A method for detecting ships reporting outside the VTS service area integrating VHF communication voice information according to claim 1, characterized in that: The detection position in S4 is specifically: For the characteristic information containing the current location, obtain its place name, search it in the VTS service area place name database, and thus detect the corresponding reporting ship outside the VTS service area; For feature information that does not contain the current location, obtain its ship name, calculate the ship name similarity with the ship name in the AIS data, and obtain the similarity ranking with the ship name in the AIS data. Then, combined with the pre-set specific threshold, screen out the ship name that meets the specific threshold and is most similar, obtain its ship point coordinates, calculate the vector from the point to each vertex of the polygonal VTS service area, and sum the angles between the vectors to determine whether it is equal to 2π within the accuracy range. Output the ship point that is not equal to 2π. The ship at this point corresponds to a ship reporting outside the VTS service area, thereby obtaining the detection result of a ship reporting outside the VTS service area.

10. A method for detecting ships reporting outside the VTS service area integrating VHF communication voice information according to claim 9, characterized in that: The ship name similarity calculation specifically includes: Calculate the overlapped character offset and non-overlapped character offset between the ship name string in the VHF corpus without the feature information of the current position and the ship name string in the AIS data; Calculate the total offset between the two ship name strings, and then get the offset similarity, that is, the ship name similarity.

Images