Digital radiographic image lettering recognition method and system

Abstract

The application provides a digital radiograph image lead character recognition method and system, comprising: acquiring a digital radiograph image, and dividing the digital radiograph image into a training set and a test set; cropping the digital radiograph image in the training set to obtain a training set cropped image; manually labeling the training set cropped image; constructing a digital radiograph image lead character recognition network; constructing a loss function of the digital radiograph image lead character recognition network; training the digital radiograph image lead character recognition network to convergence using the training set cropped image; cropping the digital radiograph image in the test set, inputting the trained digital radiograph image lead character recognition network, and obtaining a recognition result. The application can obtain multi-granularity characteristics of lead characters, effectively increase the recognition accuracy of lead characters of different sizes, use a convolutional neural network to obtain image characteristics, and solve the problem of irregular placement of lead characters by using the rotation invariance of the characteristics extracted by the convolutional neural network.

Description

Technical Field

[0001] This invention relates to the field of lead type recognition technology, specifically to a method and system for digital X-ray image lead type recognition. Background Technology

[0002] Radiographic testing is one of the most commonly used methods in non-destructive testing, widely applied in the production processes of products in important industrial fields such as aviation, aerospace, special equipment, and military. Radiographic images of products are important archival data that require long-term preservation and management. In some digital radiographic imaging applications, the default names of the generated digital radiographic images are "00001," "00002," etc., which is inconvenient for product viewing and management. Furthermore, the preservation of film generated from radiographic testing is greatly affected by external factors such as time, temperature, and humidity; it is highly susceptible to moisture damage, leading to adhesion, coating peeling, oxidation, and yellowing, resulting in the loss of original film image information. Therefore, scanners are used to convert film into digital form for storage. Currently, scanners and accompanying software on the market require manual input of information such as the film's name before scanning, a method that consumes significant labor and time costs.

[0003] Automatically recognizing the lead characters on digital radiographs and renaming them in batches based on the recognition results helps improve the efficiency of using and managing digital radiographs.

[0004] Compared to digital radiographic imaging for letterpress recognition, license plate recognition is a more mature technology. The characters on license plates are neatly arranged, and their colors differ significantly from the background. In contrast, the letterpress on products is more irregularly arranged, sometimes tilted or upside down. Furthermore, the brightness of the letterpress in the radiographic image depends heavily on the selected X-ray inspection parameters, resulting in less difference in brightness between the letterpress and the background. Therefore, directly applying license plate recognition methods to letterpress recognition yields less than ideal results.

[0005] Traditional image processing methods for recognizing lead type require manually setting numerous parameters, resulting in limited flexibility. Furthermore, varying density distributions of the objects being imaged can lead to inconsistent brightness distributions in different digital X-ray images, making traditional image processing methods less adaptable to these situations. Additionally, the uneven arrangement of the lead type significantly impacts the effectiveness of traditional image processing methods.

[0006] Therefore, there is a market need for a digital ray image lead type recognition method and system that can provide accurate recognition of lead type of different sizes. Summary of the Invention

[0007] To address the shortcomings of existing technologies, the purpose of this invention is to provide a method and system for recognizing lead type in digital X-ray images.

[0008] A method for recognizing lead type from digital X-ray images according to the present invention includes:

[0009] Step S1: Acquire digital ray images and divide the digital ray images into training and testing sets;

[0010] Step S2: Crop the digital ray images in the training set to obtain cropped training set images;

[0011] Step S3: Manually label the cropped images in the training set;

[0012] Step S4: Construct a digital ray image lead type recognition network;

[0013] Step S5: Construct the loss function for the digital ray image lead type recognition network;

[0014] Step S6: Use the cropped images from the training set to train the digital ray image lead type recognition network until convergence;

[0015] Step S7: Crop the digital ray images from the test set, input them into the trained digital ray image lead type recognition network, and obtain the recognition results.

[0016] Preferably, the method for acquiring digital radiographic images includes acquiring digital radiographic images using radiographic imaging methods, or scanning radiographic images obtained from film radiography into digital radiographic images using a scanner.

[0017] Preferably, the cropping includes reducing the length and width of the digital ray images in the training set to one-quarter of their original size, and cropping the reduced image using a sliding window with a size of W*W and a step size of S.

[0018] Preferably, the digital ray image lead type recognition network includes a shallow feature extractor, a multi-granularity feature pre-extraction module, and local, semi-global, and global deep feature extractors and position category predictors;

[0019] The shallow feature extractor includes a convolutional layer and a pooling layer;

[0020] The multi-granularity feature pre-extraction module includes three convolutional layers;

[0021] The local, semi-global, and global deep feature extraction and location category predictor includes three network branches: a local deep feature extraction and location category prediction network, a semi-global deep feature extraction and location category prediction network, and a global deep feature extraction and location category prediction network.

[0022] Preferably, the multi-granularity feature pre-extraction module consists of three convolutional layers connected in parallel;

[0023] Before the local, semi-global, and global deep feature extraction and location category predictor, the multi-granularity feature pre-extraction module first performs a multi-granularity feature extraction and fusion.

[0024] Preferably, the local deep feature extraction and location category prediction network includes a first convolutional module, a second convolutional module, a third convolutional module, and a fourth convolutional module;

[0025] The semi-global deep feature extraction and location category prediction network includes a fifth convolutional module, an upsampling module, a multi-feature connection module, and a sixth convolutional module;

[0026] The global deep feature extraction and location category prediction network includes a seventh convolutional module, an upsampling module, a multi-feature connection module, and an eighth convolutional module.

[0027] Preferably, the loss function includes a weighted classification error l cls Position error l box and confidence error l obj The calculation formula is as follows:

[0028] L = l cls +l box +l obj

[0029] Where L represents the loss function, l cls Indicates the weighted classification error, l box Indicates position error, l obj This indicates the confidence level error.

[0030] Preferably, the formula for calculating the weighted classification error is as follows:

[0031]

[0032] in, This indicates whether the i-th grid in the image to be identified contains a target. If a target exists, on the contrary, p i (c) and Let represent the true probability and predicted probability of the target belonging to the c-th category, respectively. (1-p) i (c)) α α is a hyperparameter used to add loss to misclassified samples;

[0033] The formula for calculating the position error is as follows:

[0034]

[0035] Among them, S 2λ represents the number of grids into which the image to be identified is divided, B represents the number of target prediction boxes, and λ represents the number of grids. coord This indicates the weight of coordinate error loss in the total loss function. This indicates whether a target exists in the j-th prediction box of the i-th grid in the image to be identified. If a target exists, on the contrary, x i y i w i h i These represent the coordinates of the center point of the true bounding box of the target in the image to be identified, as well as the width and height of the box. These represent the coordinates of the center point of the target bounding box in the image to be identified, as well as the width and height of the box, respectively.

[0036] The confidence error is calculated using the following formula:

[0037]

[0038] Among them, C i and These represent the true confidence level and the predicted confidence level, respectively. This indicates whether the target is not present in the j-th prediction box of the i-th grid of the image to be identified. If the target is not present, on the contrary, λ noobj This indicates the weight of the confidence loss.

[0039] Preferably, step S7 includes the following:

[0040] Step S7.1: Reduce the length and width of the digital ray images in the test set to one-quarter of the original size, and use a sliding window to crop the reduced image. The size of the sliding window is W*W, and the step size of the sliding window is S, to obtain the cropped image.

[0041] Step S7.2: Input the cropped image into the trained digital ray image lead type recognition network, and the network outputs the predicted lead type position rectangle;

[0042] Step S7.3: Set a rectangle area threshold and delete rectangles with areas below the threshold;

[0043] Step S7.4: Based on the position of the rectangle in the corresponding cropped image and the position of the cropped image in the original ray digital image, map the position of the rectangle in the cropped image to its position in the ray digital image;

[0044] Step S7.5: Delete rectangles with high overlap;

[0045] Step S7.5 includes the following:

[0046] Step S7.5.1: Sort each rectangle in the original image from highest to lowest confidence level;

[0047] Step S7.5.2: Select the rectangle with the highest confidence level, calculate the intersection-union ratio (IUR) of the rectangle with the highest confidence level with other rectangles, and determine whether the IUR of the other rectangles with the rectangle with the highest confidence level is greater than a set threshold. If so, delete the other rectangles; otherwise, proceed to step S7.5.3.

[0048] Step S7.5.3: Select the rectangle with the highest confidence among the other unprocessed rectangles, and repeat steps S7.5.1 to S7.5.2 to form the name of the ray digital image from the text in the finally retained rectangles.

[0049] A digital X-ray image lead type recognition system provided by the present invention includes:

[0050] Module M1: Acquires digital ray images and divides the digital ray images into training and testing sets;

[0051] Module M2: Crops the digital ray images in the training set to obtain cropped training set images;

[0052] Module M3: Manually labels the cropped images in the training set;

[0053] Module M4: Constructs a digital ray image lead type recognition network;

[0054] Module M5: Constructs the loss function for the digital ray image lead type recognition network;

[0055] Module M6: Train the digital ray image lead type recognition network to convergence using cropped images from the training set.

[0056] Module M7: Crops the digital ray images from the test set, inputs them into the trained digital ray image lead type recognition network, and obtains the recognition results.

[0057] Compared with the prior art, the present invention has the following beneficial effects:

[0058] 1. This invention can acquire multi-granularity features of lead type, effectively increasing the accuracy of recognizing lead type of different sizes.

[0059] 2. This invention uses a convolutional neural network to obtain image features, and can utilize the rotation invariance of the features extracted by the convolutional neural network to solve the problem of irregular placement of lead type.

[0060] 3. The multi-granularity feature pre-extraction module of this invention uses three parallel convolutional layers to extract features at different granularities, resulting in richer feature extraction. Attached Figure Description

[0061] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0062] Figure 1 This is a schematic diagram of the workflow of the method of the present invention.

[0063] Figure 2 This is a schematic diagram of the digital ray image lead type recognition network of the present invention.

[0064] Figure 3 This is a schematic diagram of the multi-granularity feature pre-extraction module of the present invention.

[0065] Figure 4 This is a diagram illustrating the recognition effect of the digital X-ray image lead type recognition method of the present invention. Detailed Implementation

[0066] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0067] According to the present invention, a method for recognizing lead type in digital X-ray images is provided, such as... Figure 1 As shown, it includes:

[0068] Step S1: Acquire digital radiographic images and divide the digital radiographic images into training and testing sets. Methods for acquiring digital radiographic images include using radiographic imaging techniques or scanning radiographic images obtained from film radiography into digital radiographic images.

[0069] Step S2: Crop the digital ray images in the training set to obtain cropped training set images. The cropping involves reducing the length and width of the digital ray images in the training set to one-quarter of their original size. A sliding window is used to crop the reduced image; the sliding window size is W*W, and the step size of the sliding window is S.

[0070] Step S3: Manually label the cropped images in the training set.

[0071] Step S4: Construct a digital ray image lead type recognition network. For example... Figure 2As shown, the digital ray image lead type recognition network includes a shallow feature extractor, a multi-granularity feature pre-extraction module, and local, semi-global, and global deep feature extractors and position category predictors.

[0072] The shallow feature extractor includes convolutional layers and pooling layers. The input to the shallow feature extractor is the cropped image, and the output is the extracted shallow image features Q.

[0073] like Figure 3 As shown, the multi-granularity feature pre-extraction module includes three convolutional layers connected in parallel. The shallow image features Q are first input to a 1x1 convolutional layer, yielding output C. Output C is then input to three convolutional layers, A1×A1, A2×A2, and A3×A3, yielding outputs C1, C2, and C3, respectively. A multi-feature connection module connects C1, C2, and C3 by channel to obtain output Q0. Compared to a single convolutional layer, the multi-granularity feature pre-extraction module uses three parallel convolutional layers to extract features at different granularities, resulting in richer feature extraction. Before the local, semi-global, and global deep feature extraction and location category predictor, the multi-granularity feature pre-extraction module performs a multi-granularity feature extraction and fusion, which facilitates subsequent extraction of deep local and global features.

[0074] The local, semi-global, and global deep feature extraction and position category predictor comprises three network branches: a local deep feature extraction and position category prediction network, a semi-global deep feature extraction and position category prediction network, and a global deep feature extraction and position category prediction network. Using three granularities for deep feature extraction and lead type position prediction enables feature extraction and recognition of lead type at different levels of granularity. This is beneficial for distinguishing lead type through obvious local features, and also allows for recognition of lead type using global features when local features are not obvious.

[0075] The local deep feature extraction and location category prediction network includes a first convolutional module, a second convolutional module, a third convolutional module, and a fourth convolutional module. The input of the local deep feature extraction and location category prediction network is the output Q0 of the multi-granularity feature pre-extraction module, and the output is the predicted feature map F1.

[0076] The semi-global deep feature extraction and location category prediction network consists of a fifth convolutional module, an upsampling module, a multi-feature connection module, and a sixth convolutional module. The semi-global deep feature extraction and location category prediction network has two inputs: the output Q1 of the first convolutional module and the output Q3 of the third convolutional module in the local deep feature extraction and location category prediction network. The output is the predicted feature map F2.

[0077] The global deep feature extraction and location category prediction network includes a seventh convolutional module, an upsampling module, a multi-feature connection module, and an eighth convolutional module. The global deep feature extraction and location category prediction network has two inputs: the output Q0 of the multi-granularity feature pre-extraction module and the output C1 of the multi-feature connection module 1 in the semi-global deep feature extraction and location category prediction network. The output is the predicted feature map F3.

[0078] Step S5: Construct the loss function for the digital ray image lead type recognition network. The loss function includes the weighted classification error l. cls Position error l box and confidence error l obj That is, L = l cls +l box +l obj .

[0079] To make the network pay more attention to misclassified samples during training, a weighted classification error is designed. Compared to the traditional classification error function, this function allows the network to focus more on misclassified samples during training. The calculation formula is as follows:

[0080]

[0081] in, This indicates whether the i-th grid in the image to be identified contains a target. If a target exists, on the contrary, p i (c) and Let represent the true probability and predicted probability of the target belonging to the c-th category, respectively. (1-p) i (c)) α This is used to add loss to misclassified samples; α is a hyperparameter.

[0082] The formula for calculating position error is as follows:

[0083]

[0084] Among them, S 2 λ represents the number of grids into which the image to be identified is divided, B represents the number of target prediction boxes, and λ represents the number of grids. coord This indicates the weight of coordinate error loss in the total loss function. This indicates whether a target exists in the j-th prediction box of the i-th grid in the image to be identified. If a target exists, on the contrary, x i y i w i h i These represent the center coordinates of the true bounding box of the target in the image to be identified, as well as the width and height of the box. These represent the coordinates of the center point of the target bounding box in the image to be identified, as well as the width and height of the box, respectively, predicted by the network.

[0085] The formula for calculating the confidence level error is as follows:

[0086]

[0087] Among them, C i and These represent the true confidence level and the predicted confidence level, respectively. This indicates whether the target is not present in the j-th prediction box of the i-th grid of the image to be identified. If the target is not present, on the contrary, λ noobj This indicates the weight of the confidence loss.

[0088] Step S6: Train the digital ray image lettering recognition network until convergence using the cropped images from the training set. In other words, train the digital ray image lettering recognition network until convergence using the cropped digital ray images from the training set.

[0089] Step S7: Cropped digital ray images from the test set are input into the trained digital ray image lead type recognition network to obtain the recognition results. Step S7 includes the following steps:

[0090] Step S7.1: Reduce the length and width of the digital ray images in the test set to one-quarter of the original size, and use a sliding window to crop the reduced image. The size of the sliding window is W*W, and the step size of the sliding window is S, to obtain the cropped image.

[0091] Step S7.2: Input the cropped image into the trained digital ray image lead type recognition network. The network outputs the predicted lead type position rectangles.

[0092] Step S7.3: Set a threshold for the area of ​​rectangles and delete rectangles with an area lower than the threshold.

[0093] Step S7.4: Based on the position of the rectangle in the corresponding cropped image and the position of the cropped image in the original ray digital image, map the position of the rectangle in the cropped image to its position in the ray digital image.

[0094] Step S7.5: Delete rectangles with high overlap. Step S7.5 includes the following steps:

[0095] Step S7.5.1: Sort each rectangle in the original image from highest to lowest confidence level. Step S7.5.2: Select the rectangle with the highest confidence level, calculate the intersection-union ratio (IU) of the highest-confidence rectangle with other rectangles, and determine if the IU of the other rectangles with the highest confidence level is greater than a set threshold. If so, delete the other rectangles; otherwise, proceed to step S7.5.3. Step S7.5.3: Select the highest-confidence rectangle from the remaining unprocessed rectangles, and repeat steps S7.5.1 to S7.5.2. The text in the finally retained rectangles forms the name of the ray digital image.

[0096] Furthermore, the implementation of the digital ray image lead type recognition method of the present invention is described in detail below with reference to the accompanying drawings:

[0097] First, the X-ray images obtained from film X-ray photography were scanned into digital X-ray images using a scanner. The resulting digital X-ray images were then divided into a training set and a test set, with 960 images in the training set and 125 images in the test set.

[0098] Then, the length and width of the digital ray images in the training set are reduced to one-quarter of their original size. A sliding window is used to crop the reduced image, with a sliding window size of 416*416 and a step size of 208, to obtain the cropped training set image.

[0099] Next, the cropped images in the training set were manually labeled, with a total of 27 label categories.

[0100] Next, a digital ray image lead type recognition network is constructed. This network consists of a shallow feature extractor, a multi-granularity feature pre-extraction module, and local, semi-global, and global deep feature extractors and position category predictors.

[0101] Specifically, such as Figure 2 As shown, the shallow feature extractor consists of 14 convolutional and pooling layers. The input to the shallow feature extractor is the cropped image, and the output is the extracted shallow image features Q. Figure 3As shown, the multi-granularity feature pre-extraction module consists of three convolutional layers. The shallow image features Q are first input into a 1x1 convolutional layer to obtain the output C. The output C is then input into 3×3, 5×5, and 7×7 convolutional layers to obtain outputs C1, C2, and C3, respectively. A multi-feature connection module connects C1, C2, and C3 by channel to obtain the output Q0. The local, semi-global, and global deep feature extraction and location category prediction network consists of three network branches: a local deep feature extraction and location category prediction network, a semi-global deep feature extraction and location category prediction network, and a global deep feature extraction and location category prediction network. The local deep feature extraction and location category prediction network consists of a first convolutional module, a second convolutional module, a third convolutional module, and a fourth convolutional module. The first convolutional module consists of 8 residual convolutional layers. The second convolutional module consists of 4 residual convolutional layers. The third convolutional module consists of 5 convolutional layers and a batch normalization layer. The fourth convolutional module consists of one convolutional layer and a batch normalization layer. The input to the local deep feature extraction and location category prediction network is the output Q0 of the multi-granularity feature pre-extraction module, and the output is the predicted feature map F1. The semi-global deep feature extraction and location category prediction network consists of a fifth convolutional module, an upsampling module, a multi-feature connection module, and a sixth convolutional module. The fifth convolutional module consists of one convolutional layer and a batch normalization layer. The sixth convolutional module consists of six convolutional layers and a batch normalization layer. The semi-global deep feature extraction and location category prediction network has two inputs: the output Q1 of the first convolutional module and the output Q3 of the third convolutional module in the local deep feature extraction and location category prediction network, and the output is the predicted feature map F2. The global deep feature extraction and location category prediction network consists of a seventh convolutional module, an upsampling module, a multi-feature connection module, and an eighth convolutional module. The seventh convolutional module consists of one convolutional layer and a batch normalization layer. The eighth convolutional module consists of one convolutional layer and a batch normalization layer. The global deep feature extraction and location category prediction network has two inputs: the output Q0 of the multi-granularity feature pre-extraction module and the output C1 of the multi-feature connection module 1 in the semi-global deep feature extraction and location category prediction network. The output is the predicted feature map F3.

[0102] Next, the loss function of the ray digital image lead type recognition network is constructed. This loss function is calculated based on the weighted classification error l. cls Position error l box and confidence error l obj That is, L = l cls +l obj +l box .

[0103] To make the network pay more attention to misclassified samples during training, a weighted classification error is designed, and the calculation formula is as follows:

[0104]

[0105] in, This indicates whether the i-th grid in the image to be identified contains a target. If a target exists, on the contrary, p i (c) and Let represent the true probability and predicted probability of the target belonging to the c-th category, respectively. (1-p) i (c)) 3 Used to increase the loss for misclassified samples.

[0106] The formula for calculating position error is as follows:

[0107]

[0108] Among them, S 2 B represents the number of grids into which the image to be identified is divided, and B represents the number of target prediction boxes. This indicates whether a target exists in the j-th prediction box of the i-th grid in the image to be identified. If a target exists, on the contrary, x i y i w i h i These represent the center coordinates of the true bounding box of the target in the image to be identified, as well as the width and height of the box. These represent the coordinates of the center point of the target bounding box in the image to be identified, as well as the width and height of the box, respectively, predicted by the network.

[0109] The formula for calculating the confidence level error is as follows:

[0110]

[0111] Among them, C i and These represent the true confidence level and the confidence level relative to the prediction, respectively. This indicates whether the target is not present in the j-th prediction box of the i-th grid of the image to be identified. If the target is not present, on the contrary,

[0112] Next, the digital ray image lead type recognition network was trained until convergence using the cropped digital ray images from the training set.

[0113] Finally, the length and width of the digital ray images in the test set are reduced to one-quarter of their original size. A sliding window is used to crop the reduced image; the sliding window size is 416*416, and the step size is 208, resulting in the cropped image. The cropped image is then input into the trained digital ray image character recognition network. The network outputs predicted bounding boxes for the character positions. A threshold of 2000 is set for the area of ​​these bounding boxes, and bounding boxes with an area less than 2000 are deleted. Based on the position of the bounding box in the corresponding cropped image and the position of the cropped image in the original digital ray image, the position of the bounding box in the cropped image is mapped to its position in the digital ray image. Redundant bounding boxes are removed using maximum suppression, as detailed below:

[0114] (1) Sort each rectangle in the original image from high to low confidence level;

[0115] (2) Select the rectangle with the highest confidence level, calculate its intersection-union ratio with other rectangles, and delete the other rectangles if the intersection-union ratio of other rectangles with the rectangle with the highest confidence level is greater than 0.5.

[0116] (3) Next, select the rectangle with the highest confidence among the other unprocessed rectangles and repeat the above process. The text in the finally retained rectangles is used to form the name of the ray digital image.

[0117] The recognition effect diagram of the digital X-ray image lead type recognition method is shown below. Figure 4 As shown.

[0118] The present invention also provides a digital ray image lead type recognition system, which can be implemented by executing the process steps of the digital ray image lead type recognition method. That is, those skilled in the art can understand the digital ray image lead type recognition method as a preferred embodiment of the digital ray image lead type recognition system.

[0119] A digital X-ray image lead type recognition system provided by the present invention includes:

[0120] Module M1: Acquires digital radiographic images and divides them into training and testing sets. Methods for acquiring digital radiographic images include using radiographic imaging techniques or scanning radiographic images obtained from film radiography into digital radiographic images.

[0121] Module M2: Crops the digital ray images in the training set to obtain cropped training set images. The cropping involves reducing the length and width of the digital ray images in the training set to one-quarter of their original size, and cropping the reduced image using a sliding window with a size of W*W and a step size of S.

[0122] Module M3: Manually labels the cropped images in the training set.

[0123] Module M4: Constructs a digital ray image character recognition network. This network includes a shallow feature extractor, a multi-granularity feature pre-extraction module, and local, semi-global, and global deep feature extraction and position category predictors. The shallow feature extractor includes convolutional layers and pooling layers. The multi-granularity feature pre-extraction module includes three convolutional layers connected in parallel. Before the local, semi-global, and global deep feature extraction and position category predictors, the multi-granularity feature pre-extraction module performs a multi-granularity feature extraction and fusion. The local, semi-global, and global deep feature extraction and position category predictors each have three network branches: a local deep feature extraction and position category prediction network, a semi-global deep feature extraction and position category prediction network, and a global deep feature extraction and position category prediction network. The local deep feature extraction and position category prediction network includes a first convolutional module, a second convolutional module, a third convolutional module, and a fourth convolutional module. The semi-global deep feature extraction and location category prediction network includes a fifth convolutional module, an upsampling module, a multi-feature connection module, and a sixth convolutional module. The global deep feature extraction and location category prediction network includes a seventh convolutional module, an upsampling module, a multi-feature connection module, and an eighth convolutional module.

[0124] Module M5: Constructs the loss function for the digital ray image lead type recognition network. The loss function includes the weighted classification error l. cls Position error l box and confidence error l obj The calculation formula is as follows:

[0125] L = l cls +l box +l obj

[0126] Where L represents the loss function, l cls Indicates the weighted classification error, l box Indicates position error, l obj This represents the confidence level error. The formula for calculating the weighted classification error is as follows:

[0127]

[0128] in, This indicates whether the i-th grid in the image to be identified contains a target. If a target exists, on the contrary, p i (c) and Let represent the true probability and predicted probability of the target belonging to the c-th category, respectively. (1-p) i (c)) α α is a hyperparameter used to add loss to misclassified samples;

[0129] The formula for calculating the position error is as follows:

[0130]

[0131] Among them, S 2 λ represents the number of grids into which the image to be identified is divided, B represents the number of target prediction boxes, and λ represents the number of grids. coord This indicates the weight of coordinate error loss in the total loss function. This indicates whether a target exists in the j-th prediction box of the i-th grid in the image to be identified. If a target exists, on the contrary, x i y i w i h i These represent the coordinates of the center point of the true bounding box of the target in the image to be identified, as well as the width and height of the box. These represent the coordinates of the center point of the target bounding box in the image to be identified, as well as the width and height of the box, respectively, predicted by the network.

[0132] The confidence error is calculated using the following formula:

[0133]

[0134] Among them, C i and These represent the true confidence level and the confidence level relative to the prediction, respectively. This indicates whether the target is not present in the j-th prediction box of the i-th grid of the image to be identified. If the target is not present, on the contrary, λ noobj This indicates the weight of the confidence loss.

[0135] Module M6: Train the digital ray image lead type recognition network to convergence using cropped images from the training set.

[0136] Module M7: Crops the digital ray images in the test set and inputs them into the trained digital ray image character recognition network to obtain the recognition results. Module M7 includes the following: Module M7.1: Reduces the length and width of the digital ray images in the test set to one-quarter of the original size, and uses a sliding window to crop the reduced image. The sliding window size is W*W, and the step size of the sliding window is S, to obtain the cropped image. Module M7.2: Inputs the cropped image into the trained digital ray image character recognition network. The network outputs the predicted character position rectangles. Module M7.3: Sets a rectangle area threshold and deletes rectangles with areas below the threshold. Module M7.4: Based on the position of the rectangle in the corresponding cropped image and the position of the cropped image in the original digital ray image, maps the position of the rectangle in the cropped image to its position in the digital ray image. Module M7.5: Deletes rectangles with high overlap. Module M7.5 includes the following: Module M7.5.1: Sorts each rectangle in the original image according to its confidence level from high to low. Module M7.5.2: Select the rectangle with the highest confidence level, calculate the intersection-union ratio (IU) of the rectangle with the highest confidence level with other rectangles, and determine whether the IU of the other rectangles with the highest confidence level is greater than a set threshold. If so, delete the other rectangles; otherwise, trigger module M7.5.3. Module M7.5.3: Select the rectangle with the highest confidence level among the other unprocessed rectangles, and repeat the triggering of modules M7.5.1 to M7.5.2. Finally, compose the text in the retained rectangles into the name of the ray digital image.

[0137] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0138] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A method for recognizing lead type from digital X-ray images, characterized in that, include: Step S1: Acquire digital ray images and divide the digital ray images into training and testing sets; Step S2: Crop the digital ray images in the training set to obtain cropped training set images; Step S3: Manually label the cropped images in the training set; Step S4: Construct a digital ray image lead type recognition network; Step S5: Construct the loss function for the digital ray image lead type recognition network; Step S6: Train the digital ray image lead type recognition network until convergence by cropping images using the training set; Step S7: Crop the digital ray images from the test set, input them into the trained digital ray image lead type recognition network, and obtain the recognition results; The digital ray image lead type recognition network includes a shallow feature extractor, a multi-granularity feature pre-extraction module, and local, semi-global, and global deep feature extractors and position category predictors; The shallow feature extractor includes a convolutional layer and a pooling layer; The multi-granularity feature pre-extraction module includes three convolutional layers; The local, semi-global, and global deep feature extraction and location category predictor includes three network branches, namely, a local deep feature extraction and location category prediction network, a semi-global deep feature extraction and location category prediction network, and a global deep feature extraction and location category prediction network. The multi-granularity feature pre-extraction module consists of three convolutional layers connected in parallel. Before the local, semi-global, and global deep feature extraction and location category predictor, the multi-granularity feature pre-extraction module first performs a multi-granularity feature extraction and fusion. The local deep feature extraction and location category prediction network includes a first convolutional module, a second convolutional module, a third convolutional module, and a fourth convolutional module, respectively. The semi-global deep feature extraction and location category prediction network includes a fifth convolutional module, an upsampling module, a multi-feature connection module, and a sixth convolutional module; The global deep feature extraction and location category prediction network includes a seventh convolutional module, an upsampling module, a multi-feature connection module, and an eighth convolutional module.

2. The digital X-ray image lead type recognition method according to claim 1, characterized in that, Methods for acquiring digital radiographic images include using radiographic imaging to acquire digital radiographic images, or using a scanner to scan radiographic images obtained from film radiography into digital radiographic images.

3. The digital X-ray image lead type recognition method according to claim 1, characterized in that, The cropping involves reducing the length and width of the digital ray images in the training set to one-quarter of their original size, and then cropping the reduced images using a sliding window with a size of W*W and a step size of S.

4. The digital X-ray image lead type recognition method according to claim 1, characterized in that, The loss function includes weighted classification error. Position error and confidence error The calculation formula is as follows: Where L represents the loss function, Indicates the weighted classification error. Indicates positional error. This indicates the confidence level error.

5. The digital X-ray image lead type recognition method according to claim 4, characterized in that, The formula for calculating the weighted classification error is as follows: in, Represents the first image to be recognized Does each grid contain a target? If so, ,on the contrary, , and These respectively indicate that the target belongs to the first... True probability and predicted probability of each category Used to increase the loss for misclassified samples. For hyperparameters; The formula for calculating the position error is as follows: in, This indicates the number of grids into which the image to be recognized is divided. Indicates the number of target prediction boxes. This indicates the weight of coordinate error loss in the total loss function. Represents the first image to be recognized The first grid Does the target exist in each prediction box? If so, ,on the contrary, , , , , These represent the coordinates of the center point of the true bounding box of the target in the image to be identified, as well as the width and height of the box. , , , These represent the coordinates of the center point of the target bounding box in the image to be identified, as well as the width and height of the box, respectively. The confidence error is calculated using the following formula: in, and These represent the true confidence level and the confidence level relative to the prediction, respectively. Represents the first image to be recognized The first grid Does the target not exist in each prediction box? If the target does not exist... ,on the contrary, , This indicates the weight of the confidence loss.

6. The digital X-ray image lead type recognition method according to claim 1, characterized in that, Step S7 includes the following: Step S7.1: Reduce the length and width of the digital ray images in the test set to one-quarter of the original size, and use a sliding window to crop the reduced image. The size of the sliding window is W*W, and the step size of the sliding window is S, to obtain the cropped image. Step S7.2: Input the cropped image into the trained digital ray image lead type recognition network, and the network outputs the predicted lead type position rectangle; Step S7.3: Set a rectangle area threshold and delete rectangles with areas below the threshold; Step S7.4: Based on the position of the rectangle in the corresponding cropped image and the position of the cropped image in the original ray digital image, map the position of the rectangle in the cropped image to its position in the ray digital image; Step S7.5: Delete rectangles with high overlap; Step S7.5 includes the following: Step S7.5.1: Sort each rectangle in the original image from highest to lowest confidence level; Step S7.5.2: Select the rectangle with the highest confidence level, calculate the intersection-union ratio (IUR) of the rectangle with the highest confidence level with other rectangles, and determine whether the IUR of the other rectangles with the rectangle with the highest confidence level is greater than a set threshold. If so, delete the other rectangles; otherwise, proceed to step S7.5.

3. Step S7.5.3: Select the rectangle with the highest confidence among the other unprocessed rectangles, and repeat steps S7.5.1 to S7.5.2 to form the name of the ray digital image from the text in the finally retained rectangles.

7. A digital ray image lead type recognition system according to any one of claims 1 to 6, characterized in that, include: Module M1: Acquires digital ray images and divides the digital ray images into training and testing sets; Module M2: Crops the digital ray images in the training set to obtain cropped training set images; Module M3: Manually labels the cropped images in the training set; Module M4: Constructs a digital ray image lead type recognition network; Module M5: Constructs the loss function for the digital ray image lead type recognition network; Module M6: Train the digital ray image lead type recognition network to convergence using cropped images from the training set; Module M7: Crops the digital ray images from the test set, inputs them into the trained digital ray image lead type recognition network, and obtains the recognition results.

Images