Railway freight train carriage counting method based on character recognition

Abstract

The invention belongs to the technical field of character recognition, and discloses a railway freight train compartment counting method based on character recognition, and the specific technical scheme is as follows: carrying out the image processing of compartment images, carrying out the rectangular marking of compartment type character and compartment number character regions, and training a train number character detection model; carrying out text labeling of compartment type characters and compartment number characters on the cut image data set, and training a vehicle number character recognition model; positioning compartment type characters and compartment number characters; reasoning the positioned compartment type characters and compartment number characters to obtain the compartment type characters and the compartment number characters; respectively calculating the similarity of the carriage type identification result and the carriage number identification result, and judging whether the two groups of identification results belong to different carriages or not to obtain the carriage number; the railway freight train carriage condition is monitored in real time, AI intelligent counting is introduced into railway freight train management, railway transportation is enabled, and the railway transportation efficiency is improved to a certain extent.

Description

Technical Field

[0001] The invention belongs to the technical field of character recognition, and in particular relates to a railway freight train carriage counting method based on character recognition. Background Art

[0002] Railway transportation has become an irreplaceable existence in the freight industry. Railway transportation has the advantages of high stability, low cost, high logistics efficiency, and large transportation volume. It can be effectively combined with various transportation modes such as sea transportation and road transportation, and has become the main force in the transportation of bulk materials. The scientific management of railway freight trains directly affects the efficiency and safety of railway transportation, and can ensure the supply speed and quality of bulk materials such as coal products. In this era of globalization, the stable management of railway freight trains, while ensuring the healthy development of the national railway transportation industry, is also of great significance to the stability of the international industrial chain supply chain.

[0003] There are mainly two existing methods for counting the number of railway freight train carriages. One is to install cameras next to the tracks for real-time monitoring or playback, which consumes manpower to count the carriages. This method has high requirements on the quality of manpower, and the functions of the cameras are not fully utilized. There are problems such as high labor cost, high counting error rate, low counting efficiency, and low equipment utilization. The other is to install sensing equipment on the tracks. When the train passes on the tracks, it will emit a sensing signal, and the number of carriages is obtained through the sensing signal. This method is highly dependent on hardware, and once there is a problem with the hardware and the correct signal cannot be emitted, the carriage counting work is abolished. It has the defects of high equipment cost, low fault tolerance, and poor stability. Summary of the invention

[0004] In order to solve the technical problems existing in the prior art, the present invention provides a railway freight train carriage counting method based on character recognition. The present invention introduces character recognition technology into railway freight train management, avoids the defects of manual counting and hardware counting, monitors the situation of railway freight train carriages in real time, empowers railway transportation, and improves railway transportation efficiency.

[0005] To achieve the above object, the technical solution adopted by the present invention is: a method for counting railway freight train carriages based on character recognition, the specific steps are:

[0006] Step 1: Collect freight train carriage image materials through a camera, perform image processing on the carriage images, perform rectangular annotation of the carriage type characters and carriage number character areas based on the processed carriage image dataset, and train a carriage number character detection model based on the YOLOv5 object detection algorithm;

[0007] Step 2: Crop the images of the carriage type characters and carriage number characters according to the labeled data of the carriage number character detection model, perform text annotation of the carriage type characters and carriage number characters on the cropped image dataset, and train the carriage number character recognition model based on the CRNN character recognition algorithm;

[0008] Step 3: Use the trained vehicle number character detection model to infer the image captured by the camera and locate the vehicle type character and vehicle number character;

[0009] Step 4: Use the trained car number character recognition model to infer the located car type characters and car number characters to obtain the car type characters and car number characters;

[0010] Step 5: Calculate the similarity of the car type and car number recognition results respectively, determine whether the two sets of recognition results belong to different cars, and obtain the number of cars.

[0011] In step one, the specific training steps are:

[0012] Step S11, collecting freight train carriage image materials through a camera;

[0013] Step S12, performing image processing on the carriage image captured by the camera, adjusting the contrast, brightness and saturation of the image after acquiring the image captured by the camera, adjusting the direction of the vehicle number characters in the image, denoising and adding noise to the image, and changing the clarity of the vehicle number characters in the image through morphological processing;

[0014] The image contrast can be adjusted as follows:

[0015] g(i,j)=af(i,j)

[0016] Among them, f(i,j) is the original pixel grayscale, g(i,j) is the adjusted pixel grayscale, and a is the coefficient;

[0017] The image brightness can be adjusted as follows:

[0018] g(i,j)=f(i,j)+b

[0019] Among them, f(i,j) is the original pixel grayscale, g(i,j) is the adjusted pixel grayscale, and b is the coefficient;

[0020] The method to adjust the image saturation is as follows:

[0021] g(i,j)=B(i,j)f(i,j)

[0022] Among them, f(i,j) is the original pixel saturation, g(i,j) is the adjusted pixel saturation, and B(i,j) is the linear transformation function for adjusting the saturation;

[0023] The methods to remove and add Gaussian noise to an image are as follows:

[0024] g(i,j)=f(i,j)±N(i,j)

[0025] Among them, f(i,j) is the original pixel grayscale, g(i,j) is the adjusted pixel grayscale, N(i,j) is the added Gaussian noise, and its probability density function is as follows:

[0026]

[0027] Where z represents the gray value, μ represents the average value or expected value of z, and σ represents the standard deviation of z;

[0028] The morphological processing of images is divided into two types: expansion and corrosion. The expansion operation is expressed by the formula:

[0029]

[0030] The corrosion operation is expressed as:

[0031]

[0032] Where element is the structural element, (x, y) is the position of the anchor point O, x' and y' are the position offsets of the pixels with structural element value 1 relative to the anchor point O, src is the original image, and dst is the result image;

[0033] Let f⊕s represent the dilation operation, and represents the erosion operation;

[0034] The opening operation is to corrode and then expand the image f with the same structural element s, which can be expressed as:

[0035]

[0036] The closing operation is to dilate and then erode the image f using the same structural element s, which can be expressed as:

[0037]

[0038] The gradient operation is the difference between the expansion and corrosion results, which can be expressed as:

[0039]

[0040] The top hat transform is the difference between the input image and its opening operation, which is expressed as:

[0041]

[0042] The black hat transform is the difference between the input image closing operation and itself, which can be expressed as:

[0043]

[0044] Affine transformation adjusts the character direction of the image. It is the superposition of linear transformation and translation transformation, which can be expressed as:

[0045]

[0046] Among them, (x, y) is the position of the anchor point O, x' and y' are the position offsets of the pixels with a structure element value of 1 relative to the anchor point O, is the rotation matrix, is the translation matrix;

[0047] Step S13: Mark the vehicle number character area with a rectangle:

[0048] Create two detection labels for carriage type and carriage number, complete the rectangular target box annotation of the character area based on the training set, and obtain a TXT text document file that can be used by the YOLOv5 model. The annotation file content contains the labels and coordinate information of each detection object in the image.

[0049] The label information in the annotation file refers to the index of the two detection label names, carriage type and carriage number, in the label array, with the indexes being 0 and 1;

[0050] There are 4 coordinate information in the annotation file, and the ratio of the horizontal coordinate of the center point of the annotation box to the image width is:

[0051] xcr=(xl+xr) / 2 / iw

[0052] The ratio of the horizontal coordinate of the center point of the annotation box to the image width:

[0053] ycr=(yt+yb) / 2 / ih

[0054] The width of the annotation box as a percentage of the image width:

[0055] bw=(xr-xl) / iw

[0056] The ratio of the height of the annotation box to the height of the image:

[0057] bh=(yb-yt) / ih

[0058] Among them, xcr is the first coordinate data, which indicates the ratio of the horizontal coordinate of the center point of the annotation box to the image width; ycr is the second coordinate data, which indicates the ratio of the horizontal coordinate of the center point of the annotation box to the image width; bw is the third coordinate data, which indicates the ratio of the width of the annotation box to the image width; bh is the fourth coordinate data, which indicates the ratio of the height of the annotation box to the image height; xl is the horizontal coordinate of the upper left corner of the annotation box, yt is the vertical coordinate of the upper left corner of the annotation box, xr is the horizontal coordinate of the lower right corner of the annotation box, and yb is the vertical coordinate of the lower right corner of the annotation box; iw is the width of the image, and ih is the height of the image;

[0059] Step S14: Train a vehicle number character detection model based on the YOLOv5 object detection algorithm to detect whether there are vehicle type or vehicle number characters in the image.

[0060] In step 2, the specific training steps are:

[0061] Step S21: cropping the images of the carriage type characters and the carriage number characters according to the annotation data of the carriage number character detection model;

[0062] According to the objective function of the document coordinate information annotated in S13, the objective function of the vehicle number character image coordinates is obtained as follows:

[0063] xl=iw*xcr-iw*bw / 2

[0064] xr=iw*xcr+iw*bw / 2

[0065] yt=ih*ycr-ih*bh / 2

[0066] yb=ih*ycr+ih*bh / 2

[0067] Among them, xl is the horizontal coordinate of the upper left corner of the annotation box, yt is the vertical coordinate of the upper left corner of the annotation box, xr is the horizontal coordinate of the lower right corner of the annotation box, and yb is the vertical coordinate of the lower right corner of the annotation box; iw is the width of the image, and ih is the height of the image; xcr is the first coordinate data in the annotation file, which indicates the ratio of the horizontal coordinate of the center point of the annotation box to the image width; ycr is the second coordinate data in the annotation file, which indicates the ratio of the horizontal coordinate of the center point of the annotation box to the image width; bw is the third coordinate data in the annotation file, which indicates the ratio of the width of the annotation box to the image width; bh is the fourth coordinate data in the annotation file, which indicates the ratio of the height of the annotation box to the image height;

[0068] After obtaining the coordinates of the upper left corner and the lower right corner of the vehicle number character annotation box, the vehicle number character image is cropped;

[0069] Step S22, annotating the cropped image data set with the carriage type characters and the carriage number characters, and saving the annotated data as a TXT text document file;

[0070] Step S23: training a vehicle number character recognition model to recognize the vehicle type or vehicle number in the vehicle number character image.

[0071] In step five, the specific steps are:

[0072] Step S51: Calculate the similarity of the car type and car number recognition results respectively. According to the car type and car number character recognition results, calculate the character similarity of adjacent recognition results. The objective function of the character similarity calculation method is as follows:

[0073] sim=(nums-diffnums) / nums*100%

[0074] Among them, sim is the character similarity of the two strings; nums is the length of the string, that is, the number of characters; diffnums is the number of characters in the two strings that have the same sort order but different characters.

[0075] Compared with the prior art, the present invention has the following specific beneficial effects:

[0076] 1. The present invention introduces character recognition technology into railway freight train management, avoids the defects of manual counting and hardware counting, monitors the situation of railway freight train carriages in real time, introduces AI intelligent counting into railway freight train management, empowers railway transportation, improves railway transportation efficiency to a certain extent, and expands the application field of AI intelligence.

[0077] 2. The present invention determines the vehicle number area by performing regional detection on the vehicle number of railway freight trains, then identifies the vehicle number according to character recognition technology, calculates the similarity of the vehicle number character string, sets a similarity threshold, and achieves the effect of distinguishing the carriages according to the character similarity.

[0078] 3. The present invention combines object detection technology, character recognition technology and character similarity calculation method to complete the statistics of the number of railway freight train carriages, provide a data basis for the railway transportation industry, and further ensure the efficient and stable operation of railway transportation.

[0079] 4. The present invention abandons the traditional method of counting carriages by manual counting or hardware signal feedback, and adopts object detection algorithm and character recognition algorithm to inject fresh blood of AI intelligence into the field of railway transportation. It has irreplaceable and significant advantages such as low cost, high counting efficiency, high fault tolerance and strong stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1The present invention is a flow chart of counting railway freight train carriages based on character recognition.

[0081] Figure 2 It is a front view of the camera and fill light installation scheme in a specific implementation method.

[0082] Figure 3 It is a top view of the camera and fill light installation scheme in a specific implementation method.

[0083] Figure 4 This is a picture of a railway freight train carriage taken by a camera in a specific implementation manner.

[0084] Figure 5 This is a diagram of railway freight train carriage types cut out in a specific implementation manner.

[0085] Figure 6 This is a railway freight train carriage numbering diagram cut out in a specific implementation manner. DETAILED DESCRIPTION

[0086] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0087] like Figure 1 The method for counting railway freight train carriages based on character recognition comprises the following specific steps:

[0088] S1: Camera and fill light installation and configuration, including:

[0089] S11: Select a suitable camera according to the running conditions of railway freight trains. The camera needs to be a network camera. Network cameras have the advantages of high definition and high resolution, support RTSP real-time streaming protocol, and can meet the needs of video image acquisition and video streaming.

[0090] In this embodiment, the selected network camera is the Hikvision 7 series intelligent network camera DS-2CD7A47EVWDV2-LZS / Q1, and the camera specifications are shown in Table 1.

[0091] Table 1 Specifications of Hikvision network cameras

[0092]

[0093]

[0094]

[0095] S12: Select appropriate fill lights according to the running conditions of railway freight trains. Fill lights can provide supplementary lighting for dark weather and night scenes, avoiding the problem of blurred images captured by cameras due to insufficient light.

[0096] In this embodiment, the selected fill light is the NVC F216 100W floodlight, and the specification parameters of the fill light are shown in Table 2.

[0097] Table 2 Specifications of NVC Fill Light

[0098] type floodlight Voltage 220V Light source type led Maximum irradiation area 50 and above Maximum Wattage 60W or more Color Temperature Cold light (above 5000K) Waterproof grade Waterproof grade Control method Light Sensitivity Control

[0099] S13: Set up the camera and fill light next to the railroad tracks.

[0100] In this embodiment, Figure 2 and Figure 3 As shown, the camera is set up on the west side of the train track. The camera and fill light installation plan is as follows:

[0101] (1) The height of the railway freight train carriage is about 3m above the ground, and the fixing rod is set up 4m from the rails;

[0102] (2) The camera is installed on the west side of the fixed pole, with the lower edge of the camera 2 meters above the ground. The camera's vertical field of view is about 50°, and the horizontal field of view can reach 10 meters;

[0103] (3) The fill light is installed on the east side of the fixed pole, with the lower edge of the fill light 1.2m above the ground. The fill light is tilted 80° upward and 80° toward the camera.

[0104] (4) The horizontal distance between the camera and the fill light is 0.5m.

[0105] S14: Configure appropriate parameters for the camera based on the on-site situation. You need to set parameters such as video resolution, video frame rate, video encoding method, image brightness, image contrast, image saturation, image sharpness, and exposure time to meet the requirements of the car number detection and recognition model training set and the needs of RTSP video stream reasoning.

[0106] In this embodiment, the camera technical parameter settings are shown in Table 3.

[0107] Table 3 Camera technical parameters

[0108]

[0109] S2: Vehicle license plate character detection model training, including:

[0110] S21: Collect freight train carriage image materials through the camera, and take screenshots through the camera's built-in preview interface, or obtain image frames through RTSP video stream;

[0111] In this embodiment, the obtained freight train car body image is shown in Figure 4 .

[0112] S22: Perform image processing on the car body image captured by the camera. After obtaining the image captured by the camera, adjust the contrast, brightness, and saturation of the image, adjust the direction of the vehicle number characters in the image, denoise the image and add noise, and change the clarity of the vehicle number characters in the image through morphological processing, which can enrich the training set materials of the vehicle number character detection model and improve the generalization performance of the model. The objective function of image processing is as follows:

[0113] (1) The method for adjusting the image contrast is as follows:

[0114] g(i,j) = af(i,j)

[0115] where f(i,j) is the original pixel gray value, g(i,j) is the adjusted pixel gray value, and the coefficient a affects the contrast of the image; when a > 1, the contrast becomes larger and the image is clearer; when a < 1, the contrast becomes smaller and the image is more blurred.

[0116] (2) The method for adjusting the image brightness is as follows:

[0117] g(i,j) = f(i,j) + b

[0118] where f(i,j) is the original pixel gray value, g(i,j) is the adjusted pixel gray value, and the coefficient b affects the brightness of the image; when b > 0, the gray value becomes larger and the whole image becomes brighter; when b < 0, the gray value becomes smaller and the whole image becomes darker.

[0119] (3) The method for adjusting the image saturation is as follows:

[0120] g(i,j) = B(i,j)f(i,j)

[0121] where f(i,j) is the original pixel saturation, g(i,j) is the adjusted pixel saturation, and B(i,j) is the linear transformation function for adjusting the saturation; when g(i,j) > f(i,j), the saturation becomes larger and the image is more vivid; when g(i,j) < f(i,j), the saturation becomes smaller and the image is more dull.

[0122] (4) The method for removing and adding Gaussian noise to the image is as follows:

[0123] g(i,j) = f(i,j) ± N(i,j)

[0124] Among them, f(i,j) is the original pixel grayscale, g(i,j) is the adjusted pixel grayscale, and N(i,j) is the added Gaussian noise. This noise is additive noise and has nothing to do with the original image signal. Gaussian noise obeys Gaussian distribution, that is, normal distribution, and its probability density function is as follows:

[0125]

[0126] Among them, z represents the gray value, μ represents the mean or expected value of z, and σ represents the standard deviation of z.

[0127] (5) The morphological processing of images is divided into two types: dilation and erosion. Furthermore, there are opening operations of erosion followed by dilation, closing operations of dilation followed by erosion, gradient operations of the difference between the results of dilation and erosion, top hat transformation of the difference between the input image and its opening operation, and black hat transformation of the difference between the closing operation of the input image and itself.

[0128] (6) The expansion operation is expressed as follows:

[0129]

[0130] The corrosion operation is expressed as:

[0131]

[0132] Where element is the structural element, (x, y) is the position of the anchor point O, x' and y' are the position offsets of the pixels with structural element value 1 relative to the anchor point O, src is the original image, and dst is the result image.

[0133] In the following content, f⊕s is used to represent the expansion operation, and Represents the erosion operation.

[0134] The opening operation is to corrode and then expand the image f with the same structural element s, which can be expressed as:

[0135]

[0136] The closing operation is to dilate and then erode the image f using the same structural element s, which can be expressed as:

[0137]

[0138] The gradient operation is the difference between the expansion and corrosion results, which can be expressed as:

[0139]

[0140] The top hat transform is the difference between the input image and its opening operation, which is expressed as:

[0141]

[0142] The black hat transform is the difference between the input image closing operation and itself, which can be expressed as:

[0143]

[0144] (6) Affine transformation can adjust the character direction of the image. It is the superposition of linear transformation and translation transformation, and is expressed by the formula:

[0145] Affine transformation can adjust the character direction of the image. It is the superposition of linear transformation and translation transformation, and is expressed by the formula:

[0146]

[0147] Among them, (x, y) is the position of the anchor point O, x' and y' are the position offsets of the pixels with a structure element value of 1 relative to the anchor point O, is the rotation matrix, is the translation matrix.

[0148] In this embodiment, according to the conditions of different carriage number image materials, the above 6 image processing methods are selectively used to change the contrast, brightness, character direction, etc. of the images taken by the camera. There are 6971 carriage number images taken by the camera, and 13853 images are obtained after image processing, achieving the purpose of enriching the training set of the carriage number character detection model.

[0149] S23: Perform rectangular annotation of the car number character area, create two detection labels: car type and car number, complete the rectangular target box annotation of the character area based on the training set, and obtain a TXT text document file that can be used by the YOLOv5 model. The annotation file content is the label and coordinate information of each detection object in the image.

[0150] The label information in the annotation file refers to the index of the two detection label names, carriage type and carriage number, in the label array, with the indexes being 0 and 1.

[0151] There are 4 coordinate information in the annotation file, and the objective functions are as follows:

[0152] (1) The ratio of the horizontal coordinate of the center point of the annotation box to the image width:

[0153] xcr=(xl+xr) / 2 / iw

[0154] (2) The ratio of the horizontal coordinate of the center point of the annotation box to the image width:

[0155] ycr=(yt+yb) / 2 / ih

[0156] (3) The ratio of the width of the annotation box to the width of the image:

[0157] bw=(xr-xl) / iw

[0158] (4) The ratio of the height of the annotation box to the height of the image:

[0159] bh=(yb-yt) / ih

[0160] Among them, xcr is the first coordinate data, which indicates the ratio of the horizontal coordinate of the center point of the annotation box to the image width; ycr is the second coordinate data, which indicates the ratio of the horizontal coordinate of the center point of the annotation box to the image width; bw is the third coordinate data, which indicates the ratio of the width of the annotation box to the image width; bh is the fourth coordinate data, which indicates the ratio of the height of the annotation box to the image height; xl is the horizontal coordinate of the upper left corner of the annotation box, yt is the vertical coordinate of the upper left corner of the annotation box, xr is the horizontal coordinate of the lower right corner of the annotation box, and yb is the vertical coordinate of the lower right corner of the annotation box; iw is the width of the image, and ih is the height of the image.

[0161] In this embodiment, the two detection tags created are shown in Table 4.

[0162] Table 4. Vehicle number character detection model labels

[0163] Tag Index Tag Name Label meaning 0 carriage_model Carriage Type 1 carriage_num Car number

[0164] The label index is two integers 0 and 1. The label carriage_model indicates the carriage type, and the label carriage_num indicates the carriage number.

[0165] The content of the annotation file for an image in the training set is shown in Table 5.

[0166] Table 5. Contents of the annotation file of the vehicle number character detection model

[0167] Tag Index Coordinate 1 Coordinate 2 Coordinate 3 Coordinate 4 0 0.26953125 0.28958333333333336 0.0546875 0.059722222222222225 1 0.3083984375 0.36944444444444446 0.134765625 0.075

[0168] In Table 5, the label indexes in the first column are 0 and 1, which represent the carriage type and carriage number respectively. The second, third, fourth and fifth columns represent the four coordinate information respectively.

[0169] S24: Train a car number character detection model based on the YOLOv5 object detection algorithm to detect whether there are car type or car number characters in the image.

[0170] In this embodiment, the YOLOv5-v7.0 object detection algorithm of the PyTorch framework is used to train the vehicle license plate character detection model. The AI ​​accelerator card used is NVIDIA GeForce RTX 3060Ti, and the pre-trained model is yolov5s.pt.

[0171] S3: Vehicle number character recognition model training, including:

[0172] S31: Crop the images of the carriage type characters and the carriage number characters according to the annotation data of the carriage number character detection model.

[0173] According to the objective function of the document coordinate information marked in S23, the objective function of the vehicle number character image coordinates is obtained as follows:

[0174] xl=iw*xcr-iw*bw / 2

[0175] xr=iw*xcr+iw*bw / 2

[0176] yt=ih*ycr-ih*bh / 2

[0177] yb=ih*ycr+ih*bh / 2

[0178] Among them, xl is the horizontal coordinate of the upper left corner of the annotation box, yt is the vertical coordinate of the upper left corner of the annotation box, xr is the horizontal coordinate of the lower right corner of the annotation box, and yb is the vertical coordinate of the lower right corner of the annotation box; iw is the width of the image, and ih is the height of the image; xcr is the first coordinate data in the annotation file, which indicates the ratio of the horizontal coordinate of the center point of the annotation box to the image width; ycr is the second coordinate data in the annotation file, which indicates the ratio of the horizontal coordinate of the center point of the annotation box to the image width; bw is the third coordinate data in the annotation file, which indicates the ratio of the width of the annotation box to the image width; bh is the fourth coordinate data in the annotation file, which indicates the ratio of the height of the annotation box to the image height.

[0179] After obtaining the coordinates of the upper left corner and the lower right corner of the vehicle number character annotation box, the vehicle number character image is cropped.

[0180] In this embodiment, according to Table 5 of the annotation file content of a certain image in the training set in S23, the actual coordinates of the vehicle number characters are obtained and shown in Table 6.

[0181] Table 6 Vehicle number character coordinates

[0182] Vehicle number type Upper left corner horizontal coordinate Upper left corner vertical coordinate Lower right corner horizontal coordinate Lower right corner vertical coordinate Carriage Type 620 374 760 460 Car number 617 478 962 586

[0183] Cropped image of carriage type see Figure 5 , the cropped carriage number image is shown in Figure 6 .

[0184] S32: Perform text annotation of the carriage type characters and the carriage number characters on the cropped image data set, and save the annotation data as a TXT text document file.

[0185] In this embodiment, Figure 5As shown in the figure, the labeling result of the car number character recognition model of the cropped car type image is: C70H; Figure 6 As shown, the labeling result of the car number character recognition model of the cropped car number image is: 1685102, and the text labels are all string types.

[0186] S33: A vehicle number character recognition model is trained based on the CRNN character recognition algorithm to identify the vehicle type or vehicle number in the vehicle number character image.

[0187] In this embodiment, the CRNN character recognition algorithm of the PyTorch framework is used to train the vehicle number character recognition model. The AI ​​accelerator card used is NVIDIA GeForce RTX 3060Ti, and the pre-trained model is crnn.pth.

[0188] S4: Car number character detection: Use the trained car number character detection model to infer the image captured by the camera and locate the car type characters and car number characters.

[0189] In this embodiment, the vehicle number detection result of a certain image is shown in Table 7.

[0190] Table 7 Vehicle number detection results

[0191] Vehicle number type Upper left corner horizontal coordinate Upper left corner vertical coordinate Lower right corner horizontal coordinate Lower right corner vertical coordinate Carriage Type 913 403 1064 479 Car number 907 467 1332 567

[0192] S5: Car number character recognition: using the trained car number character recognition model, inferring the located car type characters and car number characters to obtain the car type characters and car number characters.

[0193] In this embodiment, the recognition results of the carriage type and number of a certain image are shown in Table 8.

[0194] Table 8 Vehicle number character detection model labels

[0195] Vehicle number type Vehicle number characters Carriage Type C64K Car number 4835880

[0196] S6: Determine the number of carriages according to the similarity of the car number characters, specifically including:

[0197] S61: Calculate the similarity of the carriage type and carriage number recognition results respectively, and calculate the character similarity of adjacent recognition results based on the carriage type and number character recognition results. The objective function of the character similarity calculation method is as follows:

[0198] sim=(nums-diffnums) / nums*100%

[0199] Among them, sim is the character similarity of the two strings; nums is the length of the string, that is, the number of characters; diffnums is the number of characters in the two strings that have the same sort order but different characters.

[0200] In this embodiment, when calculating the similarity of two car type strings, the length of the string is the length of the string with more characters in the two car type strings, and the car type with the shorter string is padded with 0 at the end; when calculating the similarity of two car number strings, the length of the string is 7. When the number of characters in the identified car number is less than 7, 0 is padded at the end. When it exceeds 7, only the first 7 characters are taken.

[0201] The recognition results of the carriage type and number of a certain train are sorted out, and the string similarity between this group and the previous group of recognition results is calculated as shown in Table 9.

[0202] Table 9 The recognition results of a certain train number and its string similarity

[0203]

[0204]

[0205] S62: According to the similarity of the vehicle number recognition results, it is determined whether the two groups of vehicle number recognition results belong to different carriages.

[0206] Taking into account the influence of recognition model accuracy, two sets of vehicle numbers with exactly the same vehicle number recognition results are not considered to belong to the same carriage. If the similarity of the carriage type characters reaches 100% and the similarity of the carriage number characters reaches 85%, then the two sets of vehicle numbers are considered to belong to the same carriage.

[0207] The objective function for calculating the number of train carriages is:

[0208] cn=rn-sn

[0209] Among them, cn is the number of train carriages, rn is the number of identified carriage type and carriage number groups, and sn is the number of carriage number similarity groups whose carriage type and carriage number recognition results meet the similarity requirements.

[0210] In this embodiment, there are two groups of car number recognition results whose similarity meets the requirements that the car type character similarity is 100% and the car number character similarity reaches 85%, the number of recognition result groups is 51, and the number of similarity groups that meet the requirements is 2. Therefore, it is considered that the train has 49 cars.

[0211] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims

1. A method for counting railway freight train carriages based on character recognition, characterized in that: The specific steps are: Step 1: Collect freight train carriage image materials through a camera, perform image processing on the carriage images, perform rectangular annotation of the carriage type characters and carriage number character areas based on the processed carriage image dataset, and train a carriage number character detection model based on the YOLOv5 object detection algorithm; Step 2: Crop the images of the carriage type characters and carriage number characters according to the labeled data of the carriage number character detection model, perform text annotation of the carriage type characters and carriage number characters on the cropped image dataset, and train the carriage number character recognition model based on the CRNN character recognition algorithm; Step 3: Use the trained vehicle number character detection model to infer the image captured by the camera and locate the vehicle type character and vehicle number character; Step 4: Use the trained car number character recognition model to infer the located car type characters and car number characters to obtain the car type characters and car number characters; Step 5: Calculate the similarity of the car type and car number recognition results respectively, determine whether the two sets of recognition results belong to different cars, and obtain the number of cars.

2. A method for counting railway freight train carriages based on character recognition according to claim 1, characterized in that: In step one, the specific training steps are: Step S11, collecting freight train carriage image materials through a camera; Step S12, performing image processing on the carriage image captured by the camera, adjusting the contrast, brightness and saturation of the image after acquiring the image captured by the camera, adjusting the direction of the vehicle number characters in the image, denoising and adding noise to the image, and changing the clarity of the vehicle number characters in the image through morphological processing; The image contrast can be adjusted as follows: g(i,j)=af(i,j) Among them, f(i,j) is the original pixel grayscale, g(i,j) is the adjusted pixel grayscale, and a is the coefficient; The image brightness can be adjusted as follows: g(i,j)=f(i,j)+b Among them, f(i,j) is the original pixel grayscale, g(i,j) is the adjusted pixel grayscale, and b is the coefficient; The method to adjust the image saturation is as follows: g(i,j)=B(i,j)f(i,j) Among them, f(i,j) is the original pixel saturation, g(i,j) is the adjusted pixel saturation, and B(i,j) is the linear transformation function for adjusting the saturation; The methods to remove and add Gaussian noise to an image are as follows: g(i,j)=f(i,j)±N(i,j) Among them, f(i,j) is the original pixel grayscale, g(i,j) is the adjusted pixel grayscale, N(i,j) is the added Gaussian noise, and its probability density function is as follows: Among them, z represents the gray value, μ represents the average value or expected value of z, and σ represents the standard deviation of z; The morphological processing of images is divided into two types: expansion and corrosion. The expansion operation is expressed by the formula: The corrosion operation is expressed as: Where element is the structural element, (x, y) is the position of the anchor point O, x' and y' are the position offsets of the pixels with the structural element value of 1 relative to the anchor point O, src is the original image, and dst is the result image; Let f⊕s represent the dilation operation, and represents the erosion operation; The opening operation is to corrode and then expand the image f with the same structural element s, which can be expressed as: The closing operation is to dilate and then erode the image f using the same structural element s, which can be expressed as: The gradient operation is the difference between the expansion and corrosion results, which can be expressed as: The top hat transform is the difference between the input image and its opening operation, which is expressed as: The black hat transform is the difference between the input image closing operation and itself, which can be expressed as: Affine transformation adjusts the character direction of the image. It is the superposition of linear transformation and translation transformation, which can be expressed as: Among them, (x, y) is the position of the anchor point O, x' and y' are the position offsets of the pixels with a structure element value of 1 relative to the anchor point O, is the rotation matrix, is the translation matrix; Step S13: Mark the vehicle number character area with a rectangle: Create two detection labels for carriage type and carriage number, complete the rectangular target box annotation of the character area based on the training set, and obtain a TXT text document file that can be used by the YOLOv5 model. The annotation file content contains the labels and coordinate information of each detection object in the image. The label information in the annotation file is the index of the two detection label names, carriage type and carriage number, in the label array, with the indexes being 0 and 1; There are 4 coordinate information in the annotation file, and the ratio of the horizontal coordinate of the center point of the annotation box to the image width is: xcr=(xl+xr) / 2 / iw The ratio of the horizontal coordinate of the center point of the annotation box to the image width: ycr=(yt+yb) / 2 / ih The width of the annotation box as a percentage of the image width: bw=(xr-xl) / iw The ratio of the height of the annotation box to the height of the image: bh=(yb-yt) / ih Among them, xcr is the first coordinate data, which indicates the ratio of the horizontal coordinate of the center point of the annotation box to the image width; ycr is the second coordinate data, which indicates the ratio of the horizontal coordinate of the center point of the annotation box to the image width; bw is the third coordinate data, which indicates the ratio of the width of the annotation box to the image width; bh is the fourth coordinate data, which indicates the ratio of the height of the annotation box to the image height; xl is the horizontal coordinate of the upper left corner of the annotation box, yt is the vertical coordinate of the upper left corner of the annotation box, xr is the horizontal coordinate of the lower right corner of the annotation box, and yb is the vertical coordinate of the lower right corner of the annotation box; iw is the width of the image, and ih is the height of the image; Step S14: Train a vehicle number character detection model based on the YOLOv5 object detection algorithm to detect whether there are vehicle type or vehicle number characters in the image.

3. A method for counting railway freight train carriages based on character recognition according to claim 2, characterized in that: In step 2, the specific training steps are: Step S21: cropping the images of the carriage type characters and the carriage number characters according to the annotation data of the carriage number character detection model; According to the objective function of the document coordinate information annotated in S13, the objective function of the vehicle number character image coordinates is obtained as follows: xl=iw*xcr-iw*bw / 2 xr=iw*xcr+iw*bw / 2 yt=ih*ycr-ih*bh / 2 yb=ih*ycr+ih*bh / 2 Among them, xl is the horizontal coordinate of the upper left corner of the annotation box, yt is the vertical coordinate of the upper left corner of the annotation box, xr is the horizontal coordinate of the lower right corner of the annotation box, and yb is the vertical coordinate of the lower right corner of the annotation box; iw is the width of the image, and ih is the height of the image; xcr is the first coordinate data in the annotation file, which indicates the ratio of the horizontal coordinate of the center point of the annotation box to the image width; ycr is the second coordinate data in the annotation file, which indicates the ratio of the horizontal coordinate of the center point of the annotation box to the image width; bw is the third coordinate data in the annotation file, which indicates the ratio of the width of the annotation box to the image width; bh is the fourth coordinate data in the annotation file, which indicates the ratio of the height of the annotation box to the image height; After obtaining the coordinates of the upper left corner and the lower right corner of the vehicle number character annotation box, the vehicle number character image is cropped; Step S22, annotating the cropped image data set with the carriage type characters and the carriage number characters, and saving the annotated data as a TXT text document file; Step S23: training a vehicle number character recognition model to recognize the vehicle type or vehicle number in the vehicle number character image.

4. The method for counting railway freight train carriages based on character recognition according to claim 1, characterized in that: In step five, the specific steps are: Step S51: Calculate the similarity of the car type and car number recognition results respectively. According to the car type and car number character recognition results, calculate the character similarity of adjacent recognition results. The objective function of the character similarity calculation method is as follows: sim=(nums-diffnums) / nums*100% Among them, sim is the character similarity of the two strings; nums is the length of the string, that is, the number of characters; diffnums is the number of characters in the two strings that have the same sort order but different characters.

Images