Ship license plate recognition method, computing device and storage medium
Ship licensing is carried out through binocular cameras and deep learning technology, combining image correction and character association recognition, the accuracy of ship licensing is solved in complex environments, and the recognition effect of high accuracy and universality is achieved.
Patent Information
- Application Number
- CN202010874286.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-08-26
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2040-08-26
AI Technical Summary
The existing technology is difficult to effectively identify ship license plates, especially in complex waterway environments, long distances and uncertain weather conditions. Traditional image processing algorithms cannot handle ship number position, font size, color and arrangement of different types of ships. , resulting in low recognition accuracy.
Binocular cameras are used for image monitoring and character recognition, area recognition, image correction and character association recognition, multi-dimensional data verification is performed in combination with AIS information, and attention mechanisms and sequences are introduced to sequence models to improve recognition accuracy.
Improve the accuracy and versatility of ship licensing recognition, can effectively process ship licensing information of different fonts, sizes, colors and shapes, enhance the recognition ability in complex environments, and improve the accuracy of recognition through AIS information verification. .
Smart Images

Figure CN111985475B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of automation technology, and in particular to a ship license plate recognition method, a computing device and a storage medium. Background Art
[0002] With the rapid development of maritime trade and shipping and inland waterway transport, effective identification of ship identities, especially ship license plates, is of great significance for waterway traffic supervision, waterway transport safety, and control of coastal environmental pollution.
[0003] Artificial intelligence-related technologies such as image analysis and deep learning have made many important progress in land-based intelligent transportation systems such as license plate recognition and autonomous driving. However, the application of related technologies in the field of waterway intelligent transportation is still lacking. In recent years, many advanced methods and systems have been working on solving the problem of ship identification, mainly based on images or videos for ship detection and character positioning classification and recognition. However, considering the complex environment of the waterway, the long recognition distance and the uncertain factors such as the location, font size, and arrangement of the ship number of different types of ships at different times and weather, very few technologies can solve the problem of ship license plate recognition.
[0004] For example, since waterways, especially port waterways, often have long detection distances, ordinary cameras can only capture the rough outline of a ship. The cropped ship images have insufficient resolution, making it difficult to effectively identify the ship number.
[0005] For another example, the biggest difference between a ship license plate and a car license plate is that a ship license plate may be located anywhere on the hull, and the font size, color, shape, etc. may vary. At the same time, the ship number may be arranged in horizontal, vertical, or even curved shapes. For such situations, the use of traditional image processing algorithms can only have some effect on one or two specific types due to parameter settings, and cannot be generalized, resulting in low recognition effect and accuracy. Summary of the invention
[0006] In view of the above problems, the present invention is proposed to provide a ship license plate recognition method, computing device and storage medium that overcome the above problems or at least partially solve the above problems.
[0007] In a first aspect, the present invention provides a method for ship license plate recognition, the method comprising:
[0008] Capturing ship images through a binocular camera;
[0009] Perform regional recognition on the ship image to obtain the ship plate area image;
[0010] Perform image correction on the image of the ship plate area;
[0011] Perform character recognition on the corrected license plate area image to obtain the license plate information.
[0012] In a second aspect, the present invention provides a computing device, including: a camera, a processor, a memory, a communication interface, a communication bus, and an output unit, wherein the camera, the processor, the memory, the communication interface, and the output unit communicate with each other via the communication bus;
[0013] The memory is used to store at least one executable instruction, and the executable instruction is used to control the computing device to perform the following operations:
[0014] Capturing ship images through a binocular camera;
[0015] Perform regional recognition on the ship image to obtain the ship plate area image;
[0016] Perform image correction on the image of the ship plate area;
[0017] Perform character recognition on the corrected license plate area image to obtain the license plate information.
[0018] In a third aspect, the present invention provides a computer storage medium, wherein at least one executable instruction is stored in the computer storage medium, and the executable instruction is used to perform the following operations:
[0019] Capturing ship images through a binocular camera;
[0020] Perform regional recognition on the ship image to obtain the ship plate area image;
[0021] Perform image correction on the image of the ship plate area;
[0022] Perform character recognition on the corrected license plate area image to obtain the license plate information.
[0023] The ship license plate recognition method, computing device and storage medium provided by the present invention adopt binocular cameras to monitor the image of the waterway, one of which is responsible for full-state monitoring to find the target ship, and the other is responsible for zooming to shoot clear images of the ship. In view of the problem that the waterway monitoring distance is long and the detection target is small, the detection distance can be effectively improved, clear ship images can be captured, and video streams and pictures can be processed compatibly. The scope of application is wider, and single-point deployment can effectively cover the width of conventional waterways, up to 4 kilometers. In terms of character recognition, the ship license plate area can be comprehensively identified through character segmentation and character association, which can effectively improve the recognition rate of the ship license plate area. The accuracy of identification is improved, and an image correction step is added at the same time. In view of the problem that the ship license plate information lacks style standardization, the versatility of the identification means is improved, and the ship license plate information with different fonts, sizes, colors, shapes, and arrangements (horizontal, vertical, curved, irregular, etc.) can be effectively processed; in terms of ship license plate information recognition, a two-way recognition mechanism is adopted, the front-back correlation of characters is considered, and an attention mechanism is introduced, which can effectively improve the accuracy of character recognition; in addition, for the identified ship license plate information, the present invention further combines AIS information for data fusion, further improves the accuracy through multi-dimensional data verification means, and screens out the status of illegal ships.
[0024] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:
[0026] Figure 1 A flow chart of a ship license plate recognition method provided by an embodiment of the present invention is shown;
[0027] Figure 2 A flow chart of another ship license plate recognition method provided by an embodiment of the present invention is shown;
[0028] Figure 3 A schematic diagram of a binocular camera processing mechanism provided by an embodiment of the present invention is shown;
[0029] Figure 4 A schematic diagram of a VGG network provided by an embodiment of the present invention is shown;
[0030] Figure 5A schematic diagram of a process of image correction provided by an embodiment of the present invention is shown;
[0031] Figure 6 A schematic diagram of a BLSTM+ATTENTION network provided by an embodiment of the present invention is shown;
[0032] Figure 7 A schematic diagram of the structure of a computing device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION
[0033] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.
[0034] Embodiment 1
[0035] An embodiment of the present invention provides a ship license plate recognition method, which uses a binocular camera to monitor waterway images, performs image processing, character recognition, data storage and other processing through a server, and outputs the recognized license plate information through a human-computer interaction device. Among them, the binocular camera can be deployed at a fixed position around the waterway, and can also be deployed on the ship in a mobile scenario; the server is usually remotely deployed in the computer room or can also be on the cloud by a third party; the human-computer interaction device can be a fixed device or a portable device. Data is transmitted between the three through mobile data networks such as GSM, 3G, 4G, 5G, wireless hotspots or wired networks. It should be noted that the embodiment of the present invention is described by taking the waterway scene as an example, but it is not only applicable to the waterway scene. In actual applications, when the binocular camera is deployed on a ship, ship recognition can be performed for any specific sea area.
[0036] like Figure 1 As shown, the method of the present invention comprises:
[0037] S101. Capture a ship image through a binocular camera.
[0038] One of the cameras is used to monitor the waterway and identify ships in the waterway. When a ship is identified, the other camera is used to capture an enlarged high-definition image of the ship. In practical applications, in order to expand the monitoring range, a wide-angle camera can be used for waterway monitoring. In order to obtain an enlarged high-definition image, a zoom camera is preferably used to capture the ship image. This embodiment does not limit the brand, parameters, and specific performance of the binocular camera.
[0039] S102: Perform region recognition on the ship image to obtain a ship plate region image.
[0040] After obtaining a ship image with the overall outline of the ship as the main image, this step further identifies the license plate area from it to provide a data basis for the subsequent character recognition process. In practical applications, the license plate information is usually printed on the outer wall of the hull, but due to the lack of relevant specifications, the license plate information may appear at any position on the hull, which makes it impossible to use a simple statistical algorithm to extract the image area. This embodiment uses a deep learning network to identify and extract the license plate area image, and provides prediction data for the learning model through image feature extraction and sampling, and obtains the identified license plate area image.
[0041] S103, performing image correction on the ship plate area image.
[0042] The ship license plate information of different ships varies in font, size, color, arrangement, etc. In order to improve the versatility of the solution of this embodiment, this step adds an image correction link, especially for improving the accuracy of character recognition for special-shaped ship license plates. Image correction also uses a deep learning network. In one implementation, it can be implemented by using a positioning network + correction network cascade structure.
[0043] S104, performing character recognition on the corrected ship plate area image to obtain the ship plate information.
[0044] For the corrected character area, character recognition is performed to obtain the ship license plate information. This embodiment adopts a sequence-to-sequence model, which is composed of an encoder and a decoder, and obtains the character recognition result by prediction means. This embodiment does not limit the specific network structure of the encoder and the decoder.
[0045] Embodiment 2
[0046] Furthermore, as a refinement and improvement of the above-mentioned embodiment, the present invention also provides another ship license plate recognition method, such as Figure 2 As shown, the method includes:
[0047] S201. Monitor the waterway through wide-angle cameras and identify target ships.
[0048] The wide-angle camera monitors the waterway and transmits the captured video stream back to the server, which locates the ship in the image through object tracking and recognition algorithms, and obtains the coordinate information of the target ship and the minimum enclosing rectangle of the ship through image analysis technology. The coordinate information and the minimum enclosing rectangle are then converted into the shooting parameters of the zoom camera, such as zoom factor, lens shooting angle, etc., and sent to the zoom camera in the form of PTZ commands.
[0049] S202: When a target ship is monitored, an image of the target ship is obtained by photographing with a zoom camera.
[0050] The zoom camera changes the angle of the camera according to the PTZ command, and zooms in on the hull in the image so that the hull is located at the main position in the image. After focusing, a high-definition ship image is obtained.
[0051] The above steps S201 and S202 are as follows: Figure 3 shown.
[0052] This embodiment uses binocular cameras to perform panoramic monitoring of the waterway and high-definition capture of specific ships, which can effectively improve the resolution of ship images and provide a data basis for the subsequent accurate recognition of ship license information, and is particularly suitable for the long-distance image capture characteristics in a shipping environment.
[0053] S203, performing region recognition on the ship image through the VGG network to obtain a ship plate region image.
[0054] In this embodiment, the VGG network is used to identify the image of the ship license plate area. The VGG network has the characteristics of small convolution kernels (can be set to 1x1 to 3x3) and small pooling kernels (all 2x2 pooling kernels), and the number of layers is deeper and the feature map is wider. Because the convolution kernel focuses on expanding the number of channels and the pooling focuses on reducing the width and height, the increase in the amount of calculation is slowed down while the model architecture is deeper and wider. In addition, the VGG network also has the characteristics of full connection to convolution. The network testing phase replaces the three full connections in the training phase with three convolutions, and the test reuses the parameters during training, so that the full convolution network obtained by the test can receive inputs of any width or height because there is no restriction on full connection.
[0055] Specifically, Figure 4 As shown in the figure, the entire network consists of Conv layer, Maxpooling layer and FC layer. Conv layer is used to extract image features. If a convolution kernel is 6433, it has 64 channels, that is, 64 features will be extracted. Maxpooling layer is used to filter features according to the corresponding max or average method. At the same time, upsampling can reduce overfitting, which is conducive to deepening and widening the network. The FC layer has the largest number of parameters in the entire network. In the VGG network, the number of FC nodes has little effect on the network prediction effect.
[0056] The ship image is used as the network input for feature extraction through the Conv layer, and then the Maxpooling layer uses cross-connected upsampling operations to decode the image. Similar to the U-net structure, it can utilize more underlying features, and finally outputs two scores in the FC layer, namely the region score and the association score. The region score is the segmented character region position, and the association score is the region position associated between characters. The regional score and the association score are used to verify the obtained license plate region image, which can accurately identify the character region of any position and any arrangement. In practical applications, it can effectively identify the region of the license plate printed at any position and in any arrangement, which improves the versatility and practical value of the solution.
[0057] This embodiment takes the VGG network as an example for explanation. In practical applications, GoogLeNet or a traditional CNN network may also be used to replace the VGG network.
[0058] S204: Use a cascade network of a positioning network + a correction network to correct the image of the ship plate area.
[0059] The correction link is used to perform spatial changes on curved and irregular ship numbers and horizontal and vertical ship numbers, transforming curved and irregular text into horizontal text to reduce invalid areas. This embodiment uses a two-level cascade structure of positioning network + correction network to perform 2D transformation on the ship plate area image, such as Figure 5 As shown, firstly, the image of the ship plate area is input into the positioning network, and the key point positions of the characters in the image of the ship plate area are regressed and fitted, and then the key point positions of the characters are transformed in 2D through the correction network. Specifically:
[0060] The localization network is used to regress and fit the key point position C′ of the text in the image, C′=[c′1,...,c′ K ]∈R 2×K represents K control points, c k =[x k ,y k ] T Represents the horizontal and vertical coordinates of the Kth point.
[0061] The correction network uses Thin Plate Spline (TPS) to perform 2D transformation on the real C and C'.
[0062] The specific principles of TPS are as follows:
[0063] Known K control points {C i ,i=1,...,K}, for a given x point, the radial basis function can be used to transform the original coordinates to another coordinate:
[0064]
[0065] where σ(r) = r 2 logr is the radial basis function kernel. Each new value is affected by all other non-one-to-one control points. Written in vector form:
[0066] y k =Φ(x k )
[0067] The interpolation function can be written as:
[0068] Φ1(x)=c+a T x+w T s(x)
[0069] Where c is a scalar, a∈R 2×1 Represents 2 dimensions, w∈R N×1 , the function vector is:
[0070] s(x)=(σ(||x-x1||),σ(||x-x2||),...,σ(||xx N ||)) T
[0071] By solving, we can get:
[0072]
[0073] And further get:
[0074]
[0075] The transformation matrix can be used to map points on the original data to new data, thereby achieving the purpose of 2D transformation.
[0076] S205: Use a sequence-to-sequence model to perform character recognition to obtain the ship license plate information.
[0077] In this embodiment, a sequence-to-sequence model is used for character recognition. In one implementation, a bidirectional LSTM+attention method can be used. LSTM is called a long short-term memory network, which is a time recursive neural network suitable for processing and predicting important events with relatively long intervals and delays in time series. Sometimes the prediction may need to be determined by several previous inputs and several subsequent inputs, which will be more accurate. Therefore, a bidirectional long short-term memory network, namely BLSTM, is proposed. Compared with the recognition of a single character, the bidirectional LSTM+attention mechanism proposed in this embodiment can take into account the correlation between the characters before and after, and introduce the attention mechanism, which can effectively improve the accuracy of character recognition.
[0078] The sequence-to-sequence model consists of an encoder and a decoder. The encoder includes a convolutional network (ConvNet) and a bidirectional long short-term memory network (BLSTM). The convolutional network is used to extract features from the image of the ship plate area. The bidirectional sequence relationship of the features is analyzed through the bidirectional long short-term memory network, and multiple feature sequences of equal length are output, which are recorded as H = [h1,...,h n ],where n=ω conv .
[0079] The decoder is an attention sequence to sequence model, which is used to convert the feature sequence into a character sequence, and predict the output characters through multi-step iterative calculation to obtain the recognition result. Specifically: Assume that the attention sequence model iterates T steps and generates T-length characters, denoted as (y1,...,y T ), then at step t, the decoder is based on the encoder’s output H, the intermediate state s t-1 and the y predicted in the previous step t-1 , predict the current output y t , character or end-of-sequence symbol (EOS), and output the final prediction result as the character recognition result.
[0080] Specifically:
[0081] 1. The decoder first calculates the attention weight vector α t , the attention mechanism is expressed as:
[0082]
[0083] Where w, W, V are training weights.
[0084] 2. The attention weight indicates the importance of each feature output by the decoder. Adding the attention weight to H, we can get:
[0085]
[0086] It is used to represent a portion of the entire coded character H.
[0087] 3. RNN, based on g t , intermediate state s t-1 and the y predicted in the previous step t-1 Perform the operation and get the new state x t ,s t :
[0088] (x t ,s t )=rnn(s t-1 ,(g t ,f(yt-1 )))
[0089] Finally, use xt to predict the output of the current character:
[0090] p(y t )=softmax(W o x t +b o )
[0091] y t ~p(y t )
[0092] The above character recognition process is as follows Figure 6 shown.
[0093] S206: Acquire the AIS information of all ships within the preset sea area, and verify the identified ship license plate information according to the acquired AIS information.
[0094] The camera is connected to the host, and the host obtains the AIS information of nearby ships through the ais interface to strengthen the verification of the ship license information identified in S205. If the obtained AIS information contains the identified ship license information, it means that the identification is successful. At the same time, the status information of the ship can also be obtained from the AIS information of the ship through the association relationship of the ship license information.
[0095] In practical applications, the AIS information obtained includes but is not limited to ship identity information (type, ship name, call sign, MMSI, IMO), location, heading, speed, destination and other information. In practical applications, the preset sea area range can be set according to empirical values (for example, a 5-nautical-mile range near the camera location). The larger the range, the more ships are involved, the higher the verification success rate, and the greater the amount of calculation. In one implementation, the preset sea area range can be set in conjunction with the coverage range of the binocular camera, for example, the preset sea area range is calculated based on the angle of the wide-angle camera and the zoom multiple of the zoom camera. When the preset sea area range does not match the ship corresponding to the AIS information, the preset sea area range can be gradually expanded according to the preset step size (for example, 1 nautical mile) until the ship is successfully matched or the maximum tolerance range threshold is reached.
[0096] This step can further combine AIS information for data fusion based on accurate identification of vessel license plate information, further improve the accuracy through multi-dimensional data verification, and screen out illegal vessel status at the same time.
[0097] Embodiment 3
[0098] like Figure 7As shown, the present invention also provides a computing device, comprising: a camera, a processor, a memory, a communication interface, a communication bus and an output unit, wherein the camera, the processor, the memory, the communication interface and the output unit communicate with each other through the communication bus;
[0099] The memory is used to store at least one executable instruction, and the executable instruction is used to control the computing device to perform the following operations:
[0100] Capturing ship images through a binocular camera;
[0101] Performing region recognition on the ship image to obtain a ship plate region image;
[0102] Performing image correction on the image of the ship plate area;
[0103] Perform character recognition on the corrected license plate area image to obtain the license plate information.
[0104] Furthermore, the binocular camera is composed of a wide-angle camera and a zoom camera, and the computing device is used for:
[0105] The wide-angle camera is used to monitor the waterway and identify the target ship;
[0106] Transmitting the coordinate information of the target ship and the minimum circumscribed rectangle of the ship to the zoom camera;
[0107] The zoom camera is controlled to zoom in and focus on the ship body based on the coordinate information and the minimum circumscribed rectangle, and the ship image is obtained by shooting.
[0108] Furthermore, the computing device is used to:
[0109] Performing feature extraction on the ship image through a VGG network;
[0110] Upsample and decode the image;
[0111] Calculate and output a region score and a correlation score, wherein the region score is the segmented ship plate region position, and the correlation score is the region position of the correlation between characters in the ship plate region;
[0112] The area score and the association score are used in combination with verification to obtain the license plate area image.
[0113] Furthermore, the computing device is used to:
[0114] Inputting the ship plate area image into a positioning network, and performing regression fitting on the key point positions of the characters in the ship plate area image;
[0115] The key point positions of the text are transformed in 2D through the correction network.
[0116] Furthermore, the computing device is used to:
[0117] A sequence-to-sequence model is used for character recognition, where the sequence-to-sequence model consists of an encoder and a decoder.
[0118] Furthermore, the encoder includes a convolutional network and a bidirectional long short-term memory network. The convolutional network is used to extract features from the ship license plate area image, and the bidirectional long short-term memory network is used to analyze the bidirectional sequence relationship of the features to output multiple feature sequences of equal length.
[0119] Furthermore, the decoder is an attention sequence-to-sequence model, which is used to convert the feature sequence into a character sequence, predict the output characters through multi-step iterative calculation, and obtain the recognition result.
[0120] Furthermore, the computing device is used to:
[0121] The decoder calculates an attention weight vector;
[0122] Assigning different weight vectors to each feature in the feature sequence;
[0123] The most recent state of the current round is performed based on the weighted features, the intermediate state, and the prediction result of the previous step. After completing the preset number of iterations, the final prediction result is output as the character recognition result.
[0124] Further, the computing device is used for, after obtaining the vessel license plate information:
[0125] Obtain AIS information of all ships within the preset sea area;
[0126] Verify the identified ship license plate information based on the acquired AIS information.
[0127] Embodiment 4
[0128] The present invention also provides a non-volatile computer storage medium, which stores at least one executable instruction, and the executable instruction can execute the method in any of the above method embodiments.
[0129] The executable instructions can specifically perform the following operations:
[0130] Capturing ship images through a binocular camera;
[0131] Performing region recognition on the ship image to obtain a ship plate region image;
[0132] Performing image correction on the image of the ship plate area;
[0133] Perform character recognition on the corrected license plate area image to obtain the license plate information.
[0134] Furthermore, the binocular camera is composed of a wide-angle camera and a zoom camera, and the ship image is captured by the binocular camera, including:
[0135] The wide-angle camera is used to monitor the waterway and identify the target ship;
[0136] Transmitting the coordinate information of the target ship and the minimum circumscribed rectangle of the ship to the zoom camera;
[0137] The zoom camera is controlled to zoom in and focus on the ship body based on the coordinate information and the minimum circumscribed rectangle, and the ship image is obtained by shooting.
[0138] Furthermore, performing region recognition on the ship image to obtain a ship plate region image includes:
[0139] Performing feature extraction on the ship image through a VGG network;
[0140] Upsample and decode the image;
[0141] Calculate and output a region score and a correlation score, wherein the region score is the segmented ship plate region position, and the correlation score is the region position of the correlation between characters in the ship plate region;
[0142] The area score and the association score are used in combination with verification to obtain the license plate area image.
[0143] Furthermore, the image correction of the ship plate area image includes:
[0144] Inputting the ship plate area image into a positioning network, and performing regression fitting on the key point positions of the characters in the ship plate area image;
[0145] The key point positions of the text are transformed in 2D through the correction network.
[0146] Furthermore, the character recognition is performed on the corrected ship plate area image to obtain the ship plate information, including:
[0147] A sequence-to-sequence model is used for character recognition, where the sequence-to-sequence model consists of an encoder and a decoder.
[0148] Furthermore, the encoder includes a convolutional network and a bidirectional long short-term memory network. The convolutional network is used to extract features from the ship license plate area image, and the bidirectional long short-term memory network is used to analyze the bidirectional sequence relationship of the features to output multiple feature sequences of equal length.
[0149] Furthermore, the decoder is an attention sequence-to-sequence model, which is used to convert the feature sequence into a character sequence, predict the output characters through multi-step iterative calculation, and obtain the recognition result.
[0150] Furthermore, the step of predicting the output characters through multi-step iterative calculation to obtain the recognition result includes:
[0151] The decoder calculates an attention weight vector;
[0152] Assigning different weight vectors to each feature in the feature sequence;
[0153] The most recent state of the current round is performed based on the weighted features, the intermediate state, and the prediction result of the previous step. After completing the preset number of iterations, the final prediction result is output as the character recognition result.
[0154] Further, after obtaining the vessel license information:
[0155] Obtain AIS information of all ships within the preset sea area;
[0156] Verify the identified ship license plate information based on the acquired AIS information.
[0157] The algorithm and display provided herein are not inherently related to any particular computer, virtual system or other device. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious that the structure required for constructing such systems. In addition, the present invention is not directed to any specific programming language either. It should be understood that various programming languages can be utilized to realize the content of the present invention described herein, and the description of the above specific languages is for disclosing the best mode of the present invention.
[0158] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.
[0159] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the intention that the claimed invention requires more features than those explicitly recited in each claim. More specifically, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the present invention.
[0160] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.
[0161] In addition, those skilled in the art will appreciate that, although some embodiments herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.
[0162] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.
Claims
1. A ship license plate recognition method, characterized in that: The method comprises: Capturing ship images through a binocular camera; Performing region recognition on the ship image to obtain a ship plate region image; Performing image correction on the image of the ship plate area; Perform character recognition on the corrected license plate area image to obtain the license plate information; The binocular camera is composed of a wide-angle camera and a zoom camera, and the ship image is captured by the binocular camera, including: The wide-angle camera is used to monitor the waterway and identify the target ship; Transmitting the coordinate information of the target ship and the minimum circumscribed rectangle of the ship to the zoom camera; Based on the coordinate information and the minimum circumscribed rectangle, the zoom camera is controlled to zoom in and focus on the hull, and the image of the ship is obtained by shooting; The performing region recognition on the ship image to obtain a ship plate region image includes: Performing feature extraction on the ship image through a VGG network; Upsample and decode the image; Calculate and output a region score and a correlation score, wherein the region score is the segmented ship plate region position, and the correlation score is the region position of the correlation between characters in the ship plate region; Using the area score and the association score in combination with verification to obtain the license plate area image; The performing image correction on the ship plate area image comprises: Inputting the ship plate area image into a positioning network, and performing regression fitting on the key point positions of the characters in the ship plate area image; Performing 2D transformation on the key point positions of the text through a correction network; The step of performing character recognition on the corrected ship plate area image to obtain the ship plate information includes: A sequence-to-sequence model is used for character recognition, wherein the sequence-to-sequence model consists of an encoder and a decoder; The encoder comprises a convolutional network and a bidirectional long short-term memory network, the convolutional network is used to extract features of the ship plate area image, the bidirectional sequence relationship of the features is analyzed by the bidirectional long short-term memory network, and a plurality of feature sequences of equal length are output; The decoder is an attention sequence-to-sequence model, which is used to convert the feature sequence into a character sequence, predict the output characters through multi-step iterative calculation, and obtain the recognition result; The method of predicting the output characters through multi-step iterative calculation to obtain the recognition result includes: The decoder calculates an attention weight vector; Assigning different weight vectors to each feature in the feature sequence; The latest state of the current round is performed based on the weighted features, the intermediate state and the prediction result of the previous step. After completing the preset number of iterations, the final prediction result is output as the character recognition result; in: The decoder first calculates the attention weight vector α t , the attention mechanism is expressed as: Where w, W, V are training weights; The attention weight indicates the importance of each feature output by the decoder. By adding the attention weight to H, we can get: It is used to represent a part of the entire coded character H; RNN, based on g t , intermediate state s t-1 and the y predicted in the previous step t-1 Perform the operation and get the new state x t ,s t : (x t ,s t )=rnn(s t-1 ,(g t ,f(y t-1 ))) Finally, use xt to predict the output of the current character: p(y t )=softmax(W o x t +b o ) and t ~p(y t )。 2. The method according to claim 1, characterized in that After obtaining the vessel license information, the method further comprises: Obtain AIS information of all ships within the preset sea area; Verify the identified ship license plate information based on the acquired AIS information.
3. A computing device, characterized in that: The computing device comprises: a camera, a processor, a memory, a communication interface, a communication bus and an output unit, wherein the camera, the processor, the memory, the communication interface and the output unit communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction is used to control the computing device to perform the following operations: Capturing ship images through a binocular camera; Performing region recognition on the ship image to obtain a ship plate region image; Performing image correction on the image of the ship plate area; Perform character recognition on the corrected license plate area image to obtain the license plate information; The binocular camera is composed of a wide-angle camera and a zoom camera, and the computing device is used for: The wide-angle camera is used to monitor the waterway and identify the target ship; Transmitting the coordinate information of the target ship and the minimum circumscribed rectangle of the ship to the zoom camera; Based on the coordinate information and the minimum circumscribed rectangle, the zoom camera is controlled to zoom in and focus on the hull, and the image of the ship is obtained by shooting; The computing device is used to: Performing feature extraction on the ship image through a VGG network; Upsample and decode the image; Calculate and output a region score and a correlation score, wherein the region score is the segmented ship plate region position, and the correlation score is the region position of the correlation between characters in the ship plate region; Using the area score and the association score in combination with verification to obtain the license plate area image; The computing device is used to: Inputting the ship plate area image into a positioning network, and performing regression fitting on the key point positions of the characters in the ship plate area image; Performing 2D transformation on the key point positions of the text through a correction network; The computing device is used to: A sequence-to-sequence model is used for character recognition, wherein the sequence-to-sequence model consists of an encoder and a decoder; The encoder comprises a convolutional network and a bidirectional long short-term memory network, the convolutional network is used to extract features of the ship plate area image, the bidirectional sequence relationship of the features is analyzed by the bidirectional long short-term memory network, and a plurality of feature sequences of equal length are output; The decoder is an attention sequence-to-sequence model, which is used to convert the feature sequence into a character sequence, predict the output characters through multi-step iterative calculation, and obtain the recognition result; The computing device is used to: The decoder calculates an attention weight vector; Assigning different weight vectors to each feature in the feature sequence; The latest state of the current round is performed based on the weighted features, the intermediate state and the prediction result of the previous step. After completing the preset number of iterations, the final prediction result is output as the character recognition result; in: The decoder first calculates the attention weight vector α t , the attention mechanism is expressed as: Where w, W, V are training weights; The attention weight indicates the importance of each feature output by the decoder. By adding the attention weight to H, we can get: It is used to represent a part of the entire coded character H; RNN, based on g t , intermediate state s t-1 and the y predicted in the previous step t-1 Perform the operation and get the new state x t ,s t : (x t ,s t )=rnn(s t-1 ,(g t ,f(y t-1 ))) Finally, use xt to predict the output of the current character: p(y t )=softmax(W o x t +b o ) and t ~p(y t )。 4. The computing device according to claim 3, characterized in that The computing device is used for, after obtaining the vessel license plate information: Obtain AIS information of all ships within the preset sea area; Verify the identified ship license plate information based on the acquired AIS information.
5. A computer storage medium, wherein at least one executable instruction is stored in the computer storage medium, characterized in that: The executable instructions are used to perform the following operations: Capturing ship images through a binocular camera; Performing region recognition on the ship image to obtain a ship plate region image; Performing image correction on the image of the ship plate area; Perform character recognition on the corrected license plate area image to obtain the license plate information; The binocular camera is composed of a wide-angle camera and a zoom camera, and the ship image is captured by the binocular camera, including: The wide-angle camera is used to monitor the waterway and identify the target ship; Transmitting the coordinate information of the target ship and the minimum circumscribed rectangle of the ship to the zoom camera; Based on the coordinate information and the minimum circumscribed rectangle, the zoom camera is controlled to zoom in and focus on the hull, and the image of the ship is obtained by shooting; The performing region recognition on the ship image to obtain a ship plate region image includes: Performing feature extraction on the ship image through a VGG network; Upsample and decode the image; Calculate and output a region score and a correlation score, wherein the region score is the segmented ship plate region position, and the correlation score is the region position of the correlation between characters in the ship plate region; Using the area score and the association score in combination with verification to obtain the license plate area image; The performing image correction on the ship plate area image comprises: Inputting the ship plate area image into a positioning network, and performing regression fitting on the key point positions of the characters in the ship plate area image; Performing 2D transformation on the key point positions of the text through a correction network; The step of performing character recognition on the corrected ship plate area image to obtain the ship plate information includes: A sequence-to-sequence model is used for character recognition, wherein the sequence-to-sequence model consists of an encoder and a decoder; The encoder comprises a convolutional network and a bidirectional long short-term memory network, the convolutional network is used to extract features of the ship plate area image, the bidirectional sequence relationship of the features is analyzed by the bidirectional long short-term memory network, and a plurality of feature sequences of equal length are output; The decoder is an attention sequence-to-sequence model, which is used to convert the feature sequence into a character sequence, predict the output characters through multi-step iterative calculation, and obtain the recognition result; The method of predicting the output characters through multi-step iterative calculation to obtain the recognition result includes: The decoder calculates an attention weight vector; Assigning different weight vectors to each feature in the feature sequence; The latest state of the current round is performed based on the weighted features, the intermediate state and the prediction result of the previous step. After completing the preset number of iterations, the final prediction result is output as the character recognition result; in: The decoder first calculates the attention weight vector α t , the attention mechanism is expressed as: Where w, W, V are training weights; The attention weight indicates the importance of each feature output by the decoder. By adding the attention weight to H, we can get: It is used to represent a part of the entire coded character H; RNN, based on g t , intermediate state s t-1 and the y predicted in the previous step t-1 Perform the operation and get the new state x t ,s t : (x t ,s t )=rnn(s t-1 ,(g t ,f(y t-1 ))) Finally, use xt to predict the output of the current character: p(y t )=softmax(W o x t +b o ) and t ~p(y t )。 6. The computer storage medium according to claim 5, characterized in that After obtaining the vessel license information: Obtain AIS information of all ships within the preset sea area; Verify the identified ship license plate information based on the acquired AIS information.
