Ship name recognition method, device and electronic equipment

Abstract

The present application provides a ship name recognition method, device and electronic device. The method includes: acquiring a first image; performing ship detection on the first image to determine whether there is a ship in the first image; when there is a ship in the first image, performing text detection on the ship area of ​​the first image according to a pre-trained text line segmentation detection model to obtain a text line image; performing text recognition on the text line image according to a pre-trained ship name text recognition model to obtain a ship name recognition result, and the ship name text recognition model is obtained by multi-level transfer learning of a pre-trained general text recognition model according to a ship name data set. Since the text line segmentation detection model based on deep learning and the ship name text recognition model can jointly perform ship name recognition when there is a ship in the first image, the learning ability of the ship name text target features and the anti-interference ability of non-ship name targets can be improved, thereby improving the accuracy of the ship name recognition result.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a ship name recognition method, device and electronic equipment. Background Art

[0002] In some scenarios, it is usually necessary to identify the names of ships. For example, in maritime, fishery, inland shipping and other scenarios, video surveillance can be carried out near ports, sea channels, rivers and lakes, and the names of ships appearing in the monitoring screen can be identified. When the identified ship name is on the monitoring list, an alarm will be issued. In related technologies, when identifying the ship name, the collected image is usually input into a pre-trained model for ship name recognition to obtain the ship name recognition result. However, this method may result in misidentification and low accuracy. Summary of the invention

[0003] The present application provides a ship name recognition method, device and electronic equipment, which are used to solve the problem of low recognition accuracy of ship names in related technologies.

[0004] To solve the above technical problems, this application is implemented as follows: In a first aspect, a method for identifying a ship name is provided, comprising: acquiring a first image; Performing ship detection on the first image to determine whether there is a ship in the first image; In the case where there is a ship in the first image, performing text detection on a ship region of the first image according to a pre-trained text line segmentation detection model to obtain a text line image; Text recognition is performed on the text line image according to a pre-trained ship name text recognition model to obtain a ship name recognition result. The ship name text recognition model is obtained by performing multi-level transfer learning on a pre-trained general text recognition model according to a ship name data set.

[0005] In a second aspect, a ship name recognition device is provided, comprising: An acquisition module, acquiring a first image; a ship detection module, performing ship detection on the first image to determine whether there is a ship in the first image; a text detection module, which, when there is a ship in the first image, performs text detection on a ship region of the first image according to a pre-trained text line segmentation detection model to obtain a text line image; The ship name recognition module performs text recognition on the text line image according to a pre-trained ship name text recognition model to obtain a ship name recognition result. The ship name text recognition model is obtained by performing multi-level transfer learning on a pre-trained general text recognition model according to a ship name data set.

[0006] According to a third aspect, an electronic device is provided, including: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method as described in the first aspect.

[0007] According to a fourth aspect, a computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the method described in the first aspect.

[0008] According to a fifth aspect, a computer program product is provided, the computer program product comprising a non-transitory computer-readable storage medium storing a computer program, wherein the computer program is operable to cause a computer to execute part or all of the steps in the method described in the first aspect.

[0009] In an embodiment of the present application, after acquiring the first image, it is possible to first detect whether there is a ship in the first image. Only when there is a ship in the first image can subsequent ship name recognition be performed. Therefore, the problem of misidentification caused by the subsequent recognition object not being a ship name can be avoided; when performing ship name recognition, the text line in the ship area of ​​the image can be first detected according to the text line segmentation detection model, and then the text recognition of the detected text line image can be performed according to the ship name text recognition model, that is, the ship name recognition can be performed together based on the deep learning text line segmentation detection model and the ship name text recognition model. Therefore, the learning ability of the ship name text target features and the anti-interference ability of non-ship name targets can be improved, thereby improving the accuracy of the ship name recognition result. In addition, since the ship name text recognition model is obtained by multi-level transfer learning of the pre-trained general text recognition model based on the ship name data set, the problem of poor model effect caused by insufficient initial training data can be solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0011] Figure 1 It is a flowchart of a method for identifying a ship name according to an embodiment of the present application; Figure 2 This is a schematic diagram of the network structure of a ship name text recognition model according to an embodiment of the present application; Figure 3It is a flowchart of a method for identifying a ship name according to an embodiment of the present application; Figure 4 is a schematic diagram of a first image and a ship name recognition result in one embodiment of the present application; Figure 5 is a schematic diagram of the structure of an electronic device according to an embodiment of the present application; Figure 6 It is a structural schematic diagram of a ship name recognition device according to an embodiment of the present application. DETAILED DESCRIPTION

[0012] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in this application will be described clearly and completely below in conjunction with the drawings in one or more embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0013] The terms "first", "second", etc. in the present application and claims are used to distinguish similar objects, and are not used to describe a particular order or precedence. It should be understood that the data used in this way can be interchanged where appropriate so that the present application can be implemented in an order other than those illustrated or described herein. In addition, "and / or" in the present application and claims means at least one of the connected objects, and the character " / " generally means that the objects associated with each other are in an "or" relationship.

[0014] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0015] Figure 1 It is a flowchart of a method for identifying a ship name according to an embodiment of the present application. Figure 1 The method for identifying the ship name is shown as follows.

[0016] S12: Acquire a first image.

[0017] The first image here is an image that needs to be recognized by the ship name. In some embodiments, when acquiring the first image, an image acquired by an image acquisition device can be acquired. For example, based on maritime and inland shipping needs, cameras can be set up near ports, sea channels, and river and lake areas, and the cameras monitor these areas in real time, and capture an image at regular intervals for ship name recognition. The method of acquiring the first image is not specifically limited here.

[0018] In some embodiments, after acquiring the first image, the first image may be preprocessed to obtain an image that meets the specification, facilitate subsequent ship name recognition, and improve recognition effect. The preprocessing of the first image may include the following steps: Detecting whether the clarity of the first image meets a preset condition; When the clarity of the first image does not meet the preset condition, the first image is processed according to a preset algorithm to obtain a clear first image; Processing the size of the first image to obtain a first image with a preset size; Perform a normalization operation on the first image.

[0019] Detecting whether the clarity of the first image meets the preset conditions can be regarded as detecting the image quality of the first image to determine whether the quality of the first image meets the requirements. For example, whether the first image is blurred or has a flowery screen, whether the first image has a haze problem, etc. If the first image is blurred or has a flowery screen or has haze, it can be determined that the clarity of the first image does not meet the preset conditions, and the first image can be processed according to the preset algorithm. For example, if the first image is blurred or has a flowery screen, the image restoration algorithm can be used to repair the first image to obtain a clear and intact image. For another example, if the first image has a haze problem, the dark channel defogging algorithm can be used to process the first image to restore a clear image without haze. Optionally, if the clarity of the first image does not meet the preset conditions due to a fault in the image acquisition device, the image acquisition device can be repaired to ensure that the subsequently acquired image is clear and intact.

[0020] After obtaining a clear first image, the size of the first image can be processed to obtain a first image with a preset size. The processing of the size of the first image here can be scaling the size of the first image to scale the size of the first image to a preset size. Taking the size of the first image as h*w and the preset size of 960*960 as an example, when processing the size of the first image, the first image can be scaled according to the scaling factor r=min(960 / h,960 / w) to obtain a first image with a size of int(round(h*r)),int(round(w*r)). Among them, when scaling the first image, the Resize function in OpenCV can be used for scaling, or other tools can be used for scaling, which is not specifically limited here.

[0021] After obtaining the first image of the preset size, a normalization operation may be performed on the first image. For example, each pixel value of the first image may be divided by 255.

[0022] S14: Perform ship detection on the first image to determine whether there is a ship in the first image.

[0023] After the first image is acquired or after the first image is acquired and preprocessed, the first image may be subjected to ship name recognition. In the embodiment of the present application, when the first image is subjected to ship name recognition, the first image may be subjected to ship detection to determine whether there is a ship in the first image, so that the ship name recognition may be performed when there is a ship in the first image, thereby avoiding the problem of misidentification caused by the identification object being non-ship when performing ship name recognition.

[0024] In some embodiments, performing ship detection on the first image to determine whether there is a ship in the first image may include: Performing ship detection on the first image according to the pre-trained ship target detection model to obtain a ship detection result and a confidence level of the ship detection result; When the confidence level of the ship detection result is greater than or equal to the first threshold, it is determined that there is a ship in the first image.

[0025] The ship target detection model can be pre-trained based on sample images with ships and sample images without ships, or can be trained in other ways, which are not specifically limited here. The ship target detection model can be used to identify whether there is a ship in the image and output the identified target frame and the corresponding confidence (which can characterize the possibility or credibility of the identification result belonging to the ship). When determining whether there is a ship in the first image, the embodiment of the present application can input the first image or the pre-processed first image into the ship target detection model, and after processing by the ship target detection model, the ship identification result and the confidence of the ship identification result can be output. The ship identification result here can be the identified target frame, and the confidence can be the confidence that the target frame is a ship.

[0026] After obtaining the ship recognition result and the confidence of the ship recognition result, in order to improve the recognition accuracy, it can be further determined whether the confidence of the ship recognition result is greater than or equal to the first threshold. The first threshold can be set according to actual needs, for example, it can be 0.6, which is not specifically limited here. If the confidence of the ship recognition result is greater than or equal to the first threshold, it can be said that the possibility or credibility of the recognition result belonging to the ship is high, and it can be determined that there is a ship in the first image. Optionally, if the confidence of the ship recognition result is less than the first threshold, it can be said that the possibility or credibility of the recognition result belonging to the ship is low. At this time, in order to avoid misidentification caused by the recognition object being non-ship when the ship name is recognized in the first image later, it can be determined that there is no ship in the first image.

[0027] It should be noted that the above description takes determining whether there is a ship in the first image based on a ship target detection model as an example. In other possible implementations, other methods may be used to determine whether there is a ship in the first image. Other methods are not illustrated one by one here.

[0028] After determining whether there is a ship in the first image, if there is a ship in the first image, subsequent S16 and S18 may be executed; if there is no ship in the first image, subsequent S16 and S18 may not be executed, that is, ship name recognition is not performed.

[0029] S16: When there is a ship in the first image, text detection is performed on the ship region of the first image according to the pre-trained text line segmentation detection model to obtain a text line image.

[0030] S18: Perform text recognition on the text line image according to the pre-trained ship name text recognition model to obtain a ship name recognition result. The ship name text recognition model is obtained by performing multi-level transfer learning on a pre-trained general text recognition model according to the ship name dataset.

[0031] In the case where there is a ship in the first image, the ship name recognition can be performed on the ship area of ​​the first image to obtain a ship name recognition result. In an embodiment of the present application, when the ship name recognition is performed on the ship area of ​​the first image, text line recognition can be performed on the ship area of ​​the first image first, and then the ship name recognition is performed based on the recognized text line image, that is, text detection is first performed on the ship area of ​​the first image according to a pre-trained text line segmentation detection model to obtain a text line image, and then text recognition is performed on the text line image according to a pre-trained ship name text recognition model to obtain a ship name recognition result. In this way, since the ship name recognition can be performed together based on the deep learning text line segmentation detection model and the ship name text recognition model, the learning ability of the ship name text target features and the anti-interference ability of non-ship name targets can be improved, thereby improving the accuracy of the ship name recognition result.

[0032] In some implementations, performing text detection on the vessel region of the first image according to a pre-trained text line segmentation detection model to obtain a text line image may include: Performing text detection on the ship region of the first image according to a pre-trained text line segmentation detection model, and determining at least one text line region output by the text line segmentation detection model and a confidence level of each text line region; Determine a target text line region from at least one text line region according to the confidence level; The first image is cropped according to the target text line region to obtain a text line image corresponding to the target text line region.

[0033] The text line segmentation detection model can be pre-trained. In some embodiments, the text line segmentation detection model can adopt the DBNet model structure, and the input image is passed through the feature pyramid to obtain 4 feature maps, which are 1 / 4, 1 / 8, 1 / 16, and 1 / 32 in size, respectively. These 4 feature maps are upsampled to 1 / 4 in size and then connected to obtain the feature map F, and then the probability map P and the threshold map T can be calculated, and the final text area is obtained through a differentiable binarization operation.

[0034] When performing text detection on the ship area of ​​the first image according to the text line segmentation detection model, the first image or the ship area of ​​the first image can be input into the text line segmentation detection model, and after being processed by the text line segmentation detection model, one or more text line areas and the confidence of each text line area (which can characterize the possibility or credibility of the recognition result belonging to the text line area) can be output. Afterwards, in order to improve the recognition accuracy, the text line areas can be further screened according to the confidence of the text line areas to determine the target text line areas. Specifically, for each text line area, it can be determined whether the confidence of the text line area is greater than or equal to the second threshold value, and the second threshold value can be set according to actual needs, such as 0.6, which is not specifically limited here. If the confidence of the text line area is greater than or equal to the second threshold value, it can be said that the possibility or credibility of the recognition result belonging to the text line area is high, and the text line area can be determined as the target text line area. If the confidence of the text line area is less than the second threshold value, it can be said that the possibility or credibility of the recognition result belonging to the text line area is low, and the text line area can be filtered out. In this way, the text line areas are screened by the confidence, and one or more target text line areas can be finally determined from the recognized text line areas.

[0035] After determining the target text line area, the first image can be cropped to obtain a text line image corresponding to the target text line area, that is, to crop a text line image of the ship name that meets the requirements. The number of text line images can be the same as the number of target text line areas, that is, the number of text line images that can be cropped corresponds to the number of target text line areas.

[0036] It should be noted that, in some embodiments, after performing text detection on the ship area of ​​the first image according to the text line segmentation detection model, the detection result may be that the text line area is not detected, or the text line area is detected but after filtering the detected text line area according to the confidence level, no target text line area that meets the confidence level requirement is filtered out. In these cases, considering that the reason for these situations may be that the first image is an image taken at a long distance, the area occupied by the ship name in the first image is small and not easy to be detected and identified, therefore, in order to be able to identify the ship name in this case, the embodiment of the present application provides a solution for enlarging the first image. Specifically, in the case where the text line area or the target text line area is not detected, it may include: Taking the ship area of ​​the first image as the center, the first image is 3D-enlarged to obtain a second image; Text detection is performed on the second image according to the text line segmentation detection model to obtain a text line image.

[0037] When the first image is enlarged, the first image can be enlarged in 3D with the ship region of the first image as the center to obtain the second image. In this way, it can be ensured that the ship region in the enlarged second image is located in the middle part of the image. After obtaining the enlarged second image, text detection can be performed on the second image according to the text line segmentation detection model to obtain a text line image. The specific implementation method can refer to the above content of performing text detection on the ship region of the first image according to the text line segmentation detection model, which will not be described in detail here.

[0038] Since the ship area of ​​the first image can be 3D magnified before the text line area is detected, it is possible to recognize the ship name of an image taken at a long distance, thereby improving the recognition effect of the ship name of a small target at a long distance.

[0039] It should be noted that the above step of performing 3D magnification processing on the first image is performed when text detection is performed on the first image but no text area or target text area is detected. In some embodiments, the step of performing 3D magnification processing on the first image can also be performed before text detection is performed on the first image, that is, after determining that there is a ship in the first image, the first image is directly magnified in 3D, and then text detection is performed on the magnified first image. There is no specific limitation here on when to perform 3D magnification processing on the first image.

[0040] After performing text detection on the first image to obtain a text line image, text recognition can be performed on the text line image according to a pre-trained ship name text recognition model, that is, the text line image is input into the ship name text recognition model, and the ship name recognition result is output after being processed by the ship name text recognition model.

[0041] In the embodiment of the present application, the ship name text recognition model is a model obtained by performing multi-level transfer learning on a pre-trained general text recognition model based on a ship name data set, wherein the general text recognition model is a text recognition model applicable to a variety of text recognition scenarios. Since the training data set of the general text recognition model is easier to collect, while the ship name data set in real scenarios is more difficult to collect, therefore, the ship name text recognition model is obtained by performing multi-level transfer learning on the general text recognition model, which can solve the problem of poor model effect caused by insufficient initial training data.

[0042] When performing multi-level transfer learning on a pre-trained general text recognition model, in some embodiments, the following steps may be included: A general text recognition model is obtained by training a general text dataset; The general text recognition model is trained according to the text data set in the natural scene to obtain the text recognition model in the natural scene; The text recognition model in natural scenes is trained according to the ship name dataset to obtain the ship name text recognition model.

[0043] That is to say, when training the ship name text recognition model by multi-level transfer learning, we can first obtain a basic pre-trained model based on the general text recognition scenario (i.e. the general text recognition model mentioned above), then migrate to natural scene text recognition to obtain the next level of pre-trained model, and finally train the ship name text recognition model on the ship name dataset. The reason for this design is that there is a lot of general text data, including daily documents, web pages and other massive data containing text. At the same time, there are many related open source models available on the Internet, but the ship name text recognition scenario lacks relevant open source data and models. It is difficult to train a good ship name text recognition model relying solely on self-collected data, and multi-level transfer learning can help the ship name recognition algorithm achieve a higher accuracy rate.

[0044] The ship name dataset used in training the ship name text recognition model can be collected in advance. Considering that the existing ship name image text data is relatively small, and there are many types of ship name characters, font colors and styles, the ship name target is very small at a distance, difficult for the human eye to distinguish, and the background is complex and changeable, therefore, in order to enrich the ship name dataset and train a better ship name text recognition model, a variety of data fusion methods can be used to collect the ship name dataset.

[0045] In some embodiments, the ship name dataset may include a real scene dataset, a public dataset, and a simulated dataset, that is, the ship name dataset may be obtained by fusing the real scene dataset, the public dataset, and the simulated dataset. Among them, the real scene dataset may be obtained by collecting images of ships in real scenes and annotating images including ship names. The public dataset may be obtained by filtering open source data related to text recognition through a dictionary set, and the dictionary set is constructed based on a ship name text library, and the ship name text library includes ship names in real scenes. The simulated dataset may be a dataset generated by simulating the real scene dataset and the public dataset. In this way, by fusing the real scene dataset with other datasets to obtain the ship name dataset, the problem of insufficient real ship name data can be solved.

[0046] In the specific implementation, when collecting the ship name data set, specifically, first, a dictionary set suitable for the ship name recognition scenario can be built based on the ship name text library. The public data set is mainly crawled from the open source data related to text recognition, and then the data therein is filtered based on the dictionary set, and only the data with text in the dictionary set is retained. The real scene data set mainly comes from the video collection of inland rivers and coastal cameras, covering different scenes such as daytime, night, rainy days and foggy days, and the pictures containing ship names are selected for annotation. After obtaining the above two data sets, the frequency distribution of each character is counted. For less common characters, more supplements are required. Therefore, when generating a simulated data set, the proportion of these low-frequency characters can be increased. When generating a simulated data set, AI simulated data generation is based on PIL related library functions to render images of printed characters. The simulated generated ship name text content includes two parts, randomly selecting real ship name text strings from the ship name text library for image rendering, and randomly selecting characters from the dictionary set to splice strings for image rendering, which can fully ensure the diversity of simulated generated data. At the same time, a variety of data enhancement methods are introduced to further expand the training data.

[0047] In an embodiment of the present application, the network structure of the ship name text recognition model can be a structure improved based on the Transformer network architecture. In some embodiments, the network structure of the ship name text recognition model can include but is not limited to multiple cascaded convolutional layers, global mixing layers and local mixing layers, convolutional fusion layers, pooling layers and fully connected layers. Among them, multiple cascaded convolutional layers are used to split the input image into multiple components, and the multiple components are used to represent the stroke features of the characters. Optionally, the multiple cascaded convolutional layers can be two cascaded convolutional modules. The global mixing layer and the local mixing layer have different receptive fields for perceiving and extracting global features and local features. Optionally, the global mixing layer can be composed of a multi-head attention layer, a LayerNorm layer and a multi-layer perceptron, and the local mixing layer can use a self-attention layer with a window. The convolutional fusion layer is used for feature downsampling.

[0048] Figure 2 It is a schematic diagram of the network structure of a ship name text recognition model in one embodiment of the present application.

[0049] When performing text recognition on a text line image according to the ship name text recognition model, the text line image can be input into Figure 2 In the ship name text recognition model shown in the figure. After the text line image is input into the ship name text recognition model, it is first split into several components through two cascaded convolution modules to represent the stroke features of the characters. Then, the local and global granular features are perceived and extracted through local and global mixing layers with different receptive fields. Then, the convolution fusion layer is input to downsample the feature representation, reducing the computational cost while building a hierarchical structure. Finally, the pooling layer is used to pool the height to 1, and finally the fully connected layer, nonlinear activation layer and dropout layer are used to obtain the final feature sequence. Based on this feature sequence, a linear classifier can be used to obtain the ship name recognition result.

[0050] It should be noted that during the training phase, Figure 2 The network structure shown can use CTC loss. In the inference application stage, the ship name recognition result can be output based on CTC decoding. This can output each text content of the text line at once instead of recognizing each text piece by piece. In practical applications, the effect is better in terms of recognition accuracy and inference speed.

[0051] After obtaining the ship name recognition result, in order to further improve the recognition accuracy, it is optionally possible to determine whether the confidence of the ship name recognition result is greater than or equal to a set threshold (the ship name text recognition model can output the confidence of the ship name recognition result at the same time when outputting the ship name recognition result). If so, it can be used as the final recognition result.

[0052] In some implementations, after obtaining the ship name recognition result, the following steps may also be performed: Perform fuzzy matching on the ship name recognition result and the text list in the ship name text library; In case of successful matching, the first image and the matching ship name are output and an alarm is triggered.

[0053] That is to say, after the ship name is recognized, the ship name text string can be fuzzy matched with the text list in the ship name text library. The text list of the ship name text library is constructed by itself after collecting relevant ship name text data. If the match is successful, that is, the ship name information is matched, an alarm can be triggered to prompt that there is a ship in the list in the monitoring screen, which is convenient for subsequent management. At the same time, the visualization result of the current image frame and the recognized ship name result can also be output.

[0054] In an embodiment of the present application, after acquiring the first image, it is possible to first detect whether there is a ship in the first image. Only when there is a ship in the first image can subsequent ship name recognition be performed. Therefore, the problem of misidentification caused by the subsequent recognition object not being a ship name can be avoided; when performing ship name recognition, the text line in the ship area of ​​the image can be first detected according to the text line segmentation detection model, and then the text recognition of the detected text line image can be performed according to the ship name text recognition model, that is, the ship name recognition can be performed together based on the deep learning text line segmentation detection model and the ship name text recognition model. Therefore, the learning ability of the ship name text target features and the anti-interference ability of non-ship name targets can be improved, thereby improving the accuracy of the ship name recognition result. In addition, since the ship name text recognition model is obtained by multi-level transfer learning of the pre-trained general text recognition model based on the ship name data set, the problem of poor model effect caused by insufficient initial training data can be solved.

[0055] Figure 3 The following is a flow chart of a method for ship name recognition according to an embodiment of the present application. When performing ship name recognition, the following steps may be included.

[0056] Step 1: Acquire the first image.

[0057] Step 2: Preprocess the first image.

[0058] When preprocessing the first image, the following steps may be included: Detecting whether the clarity of the first image meets a preset condition; When the clarity of the first image does not meet the preset condition, the first image is processed according to a preset algorithm to obtain a clear first image; Processing the size of the first image to obtain a first image with a preset size; Perform a normalization operation on the first image.

[0059] Step 3: Perform ship detection on the preprocessed first image according to the ship target detection model to determine whether there is a ship in the first image.

[0060] If there is a ship in the first image, step 4 is executed; if there is no ship in the first image, the process ends.

[0061] Step 4: Perform text detection on the ship area of ​​the first image based on the pre-trained text line segmentation detection model.

[0062] In some embodiments, when it is determined that there is a ship in the first image, text detection can be performed directly on the ship region of the first image according to the pre-trained text line segmentation detection model. If a ship name text line is detected (a text line with a confidence greater than or equal to a preset threshold), step 5 is executed. If the ship name text line is not detected, the first image can be 3D-enlarged with the ship region of the first image as the center to obtain a second image. Afterwards, text detection can be performed on the second image according to the pre-trained text line segmentation detection model. If a ship name text line is detected, step 5 is executed. If the ship name text line is still not detected, the process ends.

[0063] In other embodiments, when it is determined that there is a ship in the first image, the first image can be enlarged in 3D with the ship area in the first image as the center to obtain a second image, and then text detection is performed on the second image according to a pre-trained text line segmentation detection model. If a text line with a ship name is detected, step 5 is executed; if no text line with a ship name is detected, the process ends.

[0064] Step 5: Perform text recognition on the ship name text line according to the pre-trained ship name text recognition model to obtain the ship name recognition result.

[0065] The ship name text recognition model is obtained by multi-level transfer learning of the pre-trained general text recognition model according to the ship name dataset. The ship name dataset includes a real scene dataset, a public dataset, and a simulated dataset generated based on the real scene dataset and the dataset.

[0066] The network structure of the ship name text recognition model includes but is not limited to multiple cascaded convolutional layers, global mixing layers and local mixing layers, convolutional fusion layers, pooling layers and fully connected layers. Among them, multiple cascaded convolutional layers are used to split the input image into multiple components, and the multiple components are used to represent the stroke features of the characters. The global mixing layer and the local mixing layer have different receptive fields and are used to perceive and extract global features and local features. The convolutional fusion layer is used for feature downsampling.

[0067] Step 6: Fuzzy match the ship name recognition result with the text list in the ship name text library. If the match is successful, the first image and the matching ship name are output and an alarm is triggered.

[0068] The specific implementation of the above steps 1 to 6 can be found in the above Figure 1 The specific implementation of the corresponding steps in the illustrated embodiment will not be described in detail here.

[0069] Figure 4 is a schematic diagram of a first image and a ship name recognition result according to an embodiment of the present application. Figure 4After recognizing the image shown, the ship name recognition result can be obtained: Huai'an, Suyuanxin cargo 0998.

[0070] In summary, the ship name recognition method provided in the embodiment of the present application has at least the following technical effects: (1) The algorithm is designed to first determine whether there is a ship based on hull detection. If there is a ship, its central area is magnified in 3D. The high-definition image reconstructed after the enlargement is intercepted and reconstructed for ship name text line segmentation and text content recognition. This can realize the recognition of small target ship names at a long distance.

[0071] (2) The training dataset of the ship name text recognition model is obtained by integrating AI model data generation, public dataset, and real-scene dataset. A recognition dictionary set is constructed based on the ship name text to solve the problem of insufficient real ship name data.

[0072] (3) The backbone model structure of the ship name text recognition model is innovatively optimized based on Transformer to improve the feature extraction and learning capabilities of complex ship name text. Specifically, two mixed convolutional modules with different receptive fields (i.e., global mixed layer and local mixed layer) are designed to complete the perception and extraction of local and global granular features.

[0073] (4) The ship name text recognition model is trained using a multi-level transfer learning method, which can solve the problem of poor model performance caused by insufficient initial training data.

[0074] (5) Based on the actual ship name scene effect, the appropriate text detection confidence and text recognition confidence in the inference stage are obtained to improve the accuracy of ship name recognition alarm (6) During the process of real-time video monitoring of ships, multiple images are collected to determine the image quality and select clear images for subsequent algorithm recognition. In addition, a dark channel defogging algorithm is added to the image preprocessing to improve the effect of ship name recognition in foggy weather.

[0075] (7) Ship name recognition After obtaining the ship name recognition results of all ships contained in the image, fuzzy matching is performed with the constructed ship name database to solve the problem of false alarms caused by inaccurate recognition of individual characters. The matched ship name information is organized into a structured alarm result for reporting, and the actual application effect is good.

[0076] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0077] Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 5 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0078] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0079] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0080] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a ship name recognition device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations: acquiring a first image; Performing ship detection on the first image to determine whether there is a ship in the first image; In the case where there is a ship in the first image, performing text detection on a ship region of the first image according to a pre-trained text line segmentation detection model to obtain a text line image; Text recognition is performed on the text line image according to a pre-trained ship name text recognition model to obtain a ship name recognition result. The ship name text recognition model is obtained by performing multi-level transfer learning on a pre-trained general text recognition model according to a ship name data set.

[0081] The above application Figure 5The method performed by the ship name recognition device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in this application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in this application can be directly embodied as a hardware decoding processor for execution, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0082] The electronic device may also perform Figure 1 The method is used to realize the ship name recognition device. Figure 1 The functions in the illustrated embodiments will not be described in detail in this application.

[0083] Of course, in addition to software implementation methods, the electronic device of the present application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0084] The present application also proposes a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by a portable electronic device including a plurality of application programs, enable the portable electronic device to execute Figure 1 The method of the embodiment shown is specifically used to perform the following operations: acquiring a first image; Performing ship detection on the first image to determine whether there is a ship in the first image; In the case where there is a ship in the first image, performing text detection on a ship region of the first image according to a pre-trained text line segmentation detection model to obtain a text line image; Text recognition is performed on the text line image according to a pre-trained ship name text recognition model to obtain a ship name recognition result. The ship name text recognition model is obtained by performing multi-level transfer learning on a pre-trained general text recognition model according to a ship name data set.

[0085] Figure 6 This is a schematic diagram of the structure of a ship name recognition device 60 according to an embodiment of the present application. Figure 6 In a software implementation, the ship name recognition device 60 may include: an acquisition module 61, a ship detection module 62, a text detection module 63 and a ship name recognition module 64, wherein: An acquisition module 61 is used to acquire a first image; A ship detection module 62 performs ship detection on the first image to determine whether there is a ship in the first image; a text detection module 63, which performs text detection on a ship region of the first image according to a pre-trained text line segmentation detection model to obtain a text line image when there is a ship in the first image; The ship name recognition module 64 performs text recognition on the text line image according to a pre-trained ship name text recognition model to obtain a ship name recognition result. The ship name text recognition model is obtained by performing multi-level transfer learning on a pre-trained general text recognition model according to a ship name data set.

[0086] In some implementations, after acquiring the first image, the acquisition module 61 further includes: Detecting whether the clarity of the first image meets a preset condition; When the clarity of the first image does not meet a preset condition, processing the first image according to a preset algorithm to obtain a clear first image; Processing the size of the first image to obtain a first image with a preset size; A normalization operation is performed on the first image.

[0087] In some implementations, the ship detection module 62 performs ship detection on the first image to determine whether there is a ship in the first image, including: Performing ship detection on the first image according to a pre-trained ship target detection model to obtain a ship detection result and a confidence level of the ship detection result; When the confidence level of the ship detection result is greater than or equal to a first threshold, it is determined that a ship exists in the first image.

[0088] In some implementations, the text detection module 63 performs text detection on the vessel region of the first image according to a pre-trained text line segmentation detection model to obtain a text line image, including: Performing text detection on the ship region of the first image according to a pre-trained text line segmentation detection model, and determining at least one text line region output by the text line segmentation detection model and a confidence level of each text line region; Determining a target text line region from the at least one text line region according to the confidence level; The first image is cropped according to the target text line area to obtain a text line image corresponding to the target text line area.

[0089] In some implementations, when the text detection module 63 fails to detect a text line region or a target text line region, the text detection module 63 performs 3D magnification on the first image with the ship region of the first image as the center to obtain a second image; Perform text detection on the second image according to the text line segmentation detection model to obtain a text line image.

[0090] In some embodiments, the network structure of the ship name text recognition model includes multiple cascaded convolutional layers, global mixing layers and local mixing layers, convolutional fusion layers, pooling layers and fully connected layers: The multiple cascaded convolutional layers are used to split the input image into multiple components, and the multiple components are used to represent the stroke features of the characters; The global mixing layer and the local mixing layer have different receptive fields for perceiving and extracting global features and local features; Convolutional fusion layer, used for feature downsampling.

[0091] In some embodiments, the ship name dataset includes a real scene dataset, a public dataset, and a simulated dataset generated based on the real scene dataset and the public dataset; Among them, the real scene data set is obtained by collecting images of ships in real scenes and annotating the images including ship names, and the public data set is obtained by filtering open source data related to text recognition through a dictionary set, and the dictionary set is constructed according to a ship name text library, and the ship name text library includes ship names in real scenes.

[0092] In some embodiments, after obtaining the ship name recognition result, the ship name recognition module 64 further includes: Performing fuzzy matching on the ship name recognition result and the text list in the ship name text library; If the match is successful, the first image and the matching ship name are output and an alarm is triggered.

[0093] The ship name recognition device 60 provided in the present application can also perform Figure 1 The method is to realize the ship name recognition device 60 in Figure 1 The functions of the illustrated embodiments will not be described in detail in this application.

[0094] The present application also proposes a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute part or all of the steps in the above-mentioned ship name recognition method embodiment.

[0095] In short, the above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0096] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0097] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0098] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0099] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

Claims

1. A method for identifying a ship name, characterized in that: include: acquiring a first image; Performing ship detection on the first image to determine whether there is a ship in the first image; In the case where there is a ship in the first image, performing text detection on a ship region of the first image according to a pre-trained text line segmentation detection model to obtain a text line image; Text recognition is performed on the text line image according to a pre-trained ship name text recognition model to obtain a ship name recognition result. The ship name text recognition model is obtained by performing multi-level transfer learning on a pre-trained general text recognition model according to a ship name data set.

2. The method according to claim 1, characterized in that After acquiring the first image, the method further includes: Detecting whether the clarity of the first image meets a preset condition; When the clarity of the first image does not meet a preset condition, processing the first image according to a preset algorithm to obtain a clear first image; Processing the size of the first image to obtain a first image with a preset size; A normalization operation is performed on the first image.

3. The method according to claim 1, characterized in that The performing ship detection on the first image to determine whether there is a ship in the first image includes: Performing ship detection on the first image according to a pre-trained ship target detection model to obtain a ship detection result and a confidence level of the ship detection result; When the confidence level of the ship detection result is greater than or equal to a first threshold, it is determined that a ship exists in the first image.

4. The method according to claim 1, characterized in that The performing text detection on the ship region of the first image according to the pre-trained text line segmentation detection model to obtain a text line image includes: Performing text detection on the ship region of the first image according to a pre-trained text line segmentation detection model, and determining at least one text line region output by the text line segmentation detection model and a confidence level of each text line region; Determining a target text line region from the at least one text line region according to the confidence level; The first image is cropped according to the target text line area to obtain a text line image corresponding to the target text line area.

5. The method according to claim 4, characterized in that The method further comprises: When no text line region or target text line region is detected, the first image is 3D-enlarged with the ship region of the first image as the center to obtain a second image; Perform text detection on the second image according to the text line segmentation detection model to obtain a text line image.

6. The method according to claim 1, characterized in that The network structure of the ship name text recognition model includes multiple cascaded convolutional layers, global mixing layers and local mixing layers, convolutional fusion layers, pooling layers and fully connected layers: The multiple cascaded convolutional layers are used to split the input image into multiple components, and the multiple components are used to represent the stroke features of the characters; The global mixing layer and the local mixing layer have different receptive fields for perceiving and extracting global features and local features; The convolutional fusion layer is used for feature downsampling.

7. The method according to claim 1, characterized in that The ship name data set includes a real scene data set, a public data set, and a simulated data set generated based on the real scene data set and the public data set; Among them, the real scene data set is obtained by collecting images of ships in real scenes and annotating the images including ship names, and the public data set is obtained by filtering open source data related to text recognition through a dictionary set, and the dictionary set is constructed according to a ship name text library, and the ship name text library includes ship names in real scenes.

8. The method according to claim 1, characterized in that After obtaining the ship name recognition result, the method further includes: Performing fuzzy matching on the ship name recognition result and the text list in the ship name text library; If the match is successful, the first image and the matching ship name are output and an alarm is triggered.

9. A ship name recognition device, characterized in that: include: An acquisition module, acquiring a first image; a ship detection module, performing ship detection on the first image to determine whether there is a ship in the first image; a text detection module, which, when there is a ship in the first image, performs text detection on a ship region of the first image according to a pre-trained text line segmentation detection model to obtain a text line image; The ship name recognition module performs text recognition on the text line image according to a pre-trained ship name text recognition model to obtain a ship name recognition result. The ship name text recognition model is obtained by performing multi-level transfer learning on a pre-trained general text recognition model according to a ship name data set.

10. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method according to any one of claims 1 to 8.

Images