Steel ladle number identification method, device, equipment and medium

Abstract

The invention discloses a steel ladle number identification method. The method comprises the steps that a steel ladle number main body image is detected; wherein the steel ladle number main body image is obtained through detection of a trained steel ladle number detection model; the trained text recognition model is used for recognizing the steel ladle number main body image, and text content output by the text recognition model serves as a steel ladle number recognition result; wherein a training sample for training the text recognition model is obtained by performing single-point annotation on a steel ladle number main body image, and the text recognition model is obtained based on an end-to-end text scene recognition STPOCR algorithm. According to the method, the trained steel ladle number detection model is combined with the STPOCR algorithm-based end-to-end text recognition model, the training sample is generated by utilizing single-point annotation, and parallel prediction is realized, so that the problems of high annotation cost, low processing efficiency and error accumulation of a traditional method are solved, and the accuracy of the method is improved. The method has the advantages of reducing the marking cost, improving the recognition efficiency and accuracy, and being suitable for recognition of complex arrangement steel ladle numbers.

Description

Technical Field

[0001] The present application relates to the field of image detection technology, and in particular to a ladle number recognition method, device, equipment and medium. Background Art

[0002] In the context of ladle number character recognition, the structure and arrangement of numbers vary widely, encompassing both linear and nonlinear character arrangements. These different number arrangements present different labeling and recognition requirements. Traditional labeling methods often rely on manual operations, such as using horizontal rectangles, rotated rectangles, quadrilaterals, and polygons, resulting in high costs. Summary of the Invention

[0003] In view of the above-mentioned shortcomings of the prior art, the present application provides a ladle number identification method, device, equipment and medium to solve at least one defect in the prior art.

[0004] To achieve the above and other purposes, the present application provides a ladle number identification method, which includes:

[0005] Detecting the ladle number main body image; wherein the ladle number main body image is obtained by detecting the trained ladle number detection model;

[0006] The ladle number main body image is recognized using a trained text recognition model, and the text content output by the text recognition model is used as the ladle number recognition result; wherein, the training samples for training the text recognition model are obtained by single-point annotation of the ladle number main body image, and the text recognition model is obtained based on the end-to-end text scene recognition STPOCR algorithm.

[0007] In one embodiment of the present application, the method of using a trained text recognition model to recognize the main image of the ladle number includes:

[0008] Performing feature extraction on the main image of the ladle number through the feature extraction module in the text recognition model to obtain a continuous text sequence;

[0009] Converting the continuous text sequence into a discretized text sequence through a discretization module in the text recognition model, and identifying one or more text instances in the discretized text sequence;

[0010] Predicting the center point positions of the one or more text instances by using an instance allocation decoder in the text recognition model to obtain the center point positions of the one or more text instances;

[0011] The parallel recognition decoder in the text recognition model predicts the ladle number corresponding to each text instance in parallel according to the text instance and the center point position of the text instance.

[0012] In one embodiment of the present application, the method for obtaining the center point position of the text instance includes:

[0013] Inserting the first tag into the head of the text instance and inserting the second tag into the tail of the text instance to obtain a first text instance;

[0014] Detecting the coordinates (xi, yi) of the center point of the first text instance;

[0015] The coordinates of the center point of the text instance are discretized into [1, n bins ], where n bins Control the degree of dispersion;

[0016] The discretized coordinates are used to represent the position features of the text instance and the center point position of the text instance is obtained.

[0017] In one embodiment of the present application, the method for obtaining the continuous text sequence includes:

[0018] Performing feature extraction on the main image of the ladle number through a convolutional neural network to obtain a first feature map;

[0019] The first feature map is input into the Transformer encoder to obtain a text sequence.

[0020] In one embodiment of the present application, the loss function in the text recognition model training process is:

[0021]

[0022] Where L is the sequence length, w i is the likelihood weight of the i-th mark, For input S 1:i Under the condition that the input image I has the output sequence The probability of , S is the input sequence.

[0023] In one embodiment of the present application, before the step of parallelly predicting the ladle number corresponding to each text instance based on the text instance and the center point position of the text instance by the parallel recognition decoder in the text recognition model, the method further includes:

[0024] Extract hidden text instance position features and corresponding text position prediction results;

[0025] Converting the text position prediction result into embedded data;

[0026] adding the embedded data into the text instance position feature.

[0027] In an embodiment of the present application, in the process of predicting the ladle number corresponding to each text instance in parallel, the method further comprises:

[0028] filling or truncating the text instance to a fixed length K and adding a blank mark to fill the blank of the short text instance.

[0029] To achieve the above object and other related objects, the present application provides a ladle number identification device, which comprises:

[0030] an image detection module, configured to detect a ladle number main image; wherein the ladle number main image is detected by a trained ladle number detection model;

[0031] an image recognition module, configured to recognize the ladle number main image by using a trained text recognition model, and take the text content output by the text recognition model as the ladle number identification result; wherein the training sample of the text recognition model is obtained by single-point annotation on the ladle number main image, and the text recognition model is obtained based on a scene text and picture optical character recognition (STPOCR) algorithm.

[0032] To achieve the above object and other related objects, the present application provides a ladle number identification device, which comprises:

[0033] one or more processors; and

[0034] a memory, configured to store one or more programs, when the one or more programs are executed by the one or more processors, the memory enables the ladle number identification method.

[0035] To achieve the above object and other related objects, the present application provides one or more machine readable media, which store instructions, when executed by one or more processors, enable the processors to execute the ladle number identification method.

[0036] The present application has the following beneficial effects:

[0037] The present application provides a ladle number recognition method, comprising: detecting the main image of the ladle number; wherein, the main image of the ladle number is obtained by detecting a trained ladle number detection model; utilizing a trained text recognition model to recognize the main image of the ladle number, and using the text content output by the text recognition model as the ladle number recognition result; wherein, the training samples for training the text recognition model are obtained by single-point annotation of the main image of the ladle number, and the text recognition model is obtained based on the end-to-end text scene recognition STPOCR algorithm. The present application combines a trained ladle number detection model with an end-to-end text recognition model based on the STPOCR algorithm, utilizes single-point annotation to generate training samples and realizes parallel prediction, thereby solving the problems of high labeling cost, low processing efficiency, and error accumulation of traditional methods, and has the advantages of reducing labeling cost, improving recognition efficiency and accuracy, and adapting to complex arrangement ladle number recognition.

[0038] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings are incorporated into and constitute a part of the specification, illustrating embodiments consistent with the present application and, together with the specification, serving to explain the principles of the present application. It is obvious that the drawings described below are merely some embodiments of the present application, and a person of ordinary skill in the art can derive other drawings based on these drawings without inventive effort. In the drawings:

[0040] Figure 1 This is a flow chart of a ladle number identification method according to an embodiment of the present application;

[0041] Figure 2 This is a flow chart for identifying a ladle number main body image in one embodiment of the present application;

[0042] Figure 3 A flowchart of a method for obtaining the center point position of a text instance according to an embodiment of the present application;

[0043] Figure 4 This is a block diagram of a ladle number recognition device according to an embodiment of the present application;

[0044] Figure 5 A schematic diagram of the structure of a computer system suitable for implementing the memory of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0045] The following describes the embodiments of the present application through specific examples. Those skilled in the art can easily understand the other advantages and effects of the present application from the content disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0046] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application. Therefore, the illustrations only show components related to the present application and are not drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component can be changed at will, and the component layout type may also be more complicated.

[0047] Although the terms "first," "second," "A," and "B," etc. may be used herein to describe various elements, these elements should not be limited by these terms and are merely used to distinguish one element from another. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element without departing from the scope of the technology described below. The term "and / or" includes a combination of a plurality of related items or any of the plurality of related items.

[0048] As used herein, unless the context indicates otherwise, the singular form is intended to include the plural form, and it will be understood that the term "comprising" means the presence of stated features, quantities, steps, operations, elements, or combinations thereof, but does not preclude the presence or addition of one or more other features, quantities, steps, operations, elements, components, or combinations thereof.

[0049] Before describing the components in detail, it is intended to clarify that the components in this specification are divided only by the primary function of each component. That is, two or more components described below may be combined into one component, or may be divided into two or more components based on more detailed functions. In addition to the primary function of the component, each component described below may also perform some or all of the functions of other components, and some of the primary functions of each component may be exclusively performed by other components.

[0050] The embodiments of the present application respectively propose a ladle number recognition method, a ladle number recognition device, a ladle number recognition equipment, and a computer-readable storage medium, which will be described in detail below.

[0051] See also Figure 1 , Figure 1 This is a flow chart of a ladle number identification method according to one embodiment of the present application. Figure 1As shown, the ladle number identification method includes steps S110 to S120:

[0052] Step S110, detecting the main image of the ladle number; wherein the main image of the ladle number is obtained by detecting the ladle number through a trained ladle number detection model;

[0053] The ladle number detection model is a deep learning model used to locate the image region where the ladle number is located. Specifically, the ladle number detection model locates the region in the input image and outputs an image of the main area containing the complete ladle number.

[0054] The ladle number detection model can be trained using an initial ladle number detection model using a ladle number detection dataset. The ladle number detection dataset includes multiple ladle number detection images, each of which includes the ladle number itself and linearly or / and nonlinearly arranged characters. The ladle number detection images are captured using a camera. It should be noted that the camera is positioned horizontally on the front of the character to be recognized within the subject, with the angle between the camera and the character recognition surface being between 45° and 90°.

[0055] Among them, the method of training the ladle number detection model includes:

[0056] Capture ladle number detection images in specific industrial scenarios and perform image annotation;

[0057] Use the rectangular selection box of the image annotation tool to mark the location information of the ladle number body to create a ladle number detection dataset;

[0058] The ladle number detection dataset is divided into three parts: training set, test set, and validation set;

[0059] The ladle number detection model is trained using the data in the training set. By learning the target features within the labeled box of each ladle number image in the training set, the ladle number detection model is finally obtained.

[0060] The ladle number detection model is tested using the test set, and verified using the validation set, and finally the ladle number detection model is obtained.

[0061] It should be noted that the ladle number detection dataset provides training information including basic image attributes and annotation information. Basic image attributes include filename, width, height, and depth; annotation information includes the following: x1, y1, x2, y2, x3, y3, x4, and y4 coordinates of the upper left corner of the text; and class, the class of the target object.

[0062] When it is necessary to detect the main image of the ladle number, the trained ladle number detection model is started to detect the real-time ladle number image to obtain the main image of the ladle number.

[0063] Step S120, use the trained text recognition model to recognize the main image of the ladle number, and use the text content output by the text recognition model as the ladle number recognition result; wherein, the training sample for training the text recognition model is obtained by single-point annotation of the main image of the ladle number, and the text recognition model is obtained based on the end-to-end text scene recognition STPOCR algorithm.

[0064] The text recognition model is an end-to-end recognition model based on sequence prediction. Specifically, the text detection model can be based on the Yolov5n neural network. Of course, other object recognition neural networks can also be used for the text recognition model.

[0065] Single-point annotation refers to a method of marking only the center position of the ladle number, which can be achieved by marking the coordinates of the geometric center point of the text instance in the image.

[0066] The STPOCR algorithm, which stands for Scene Text Parsing Optical Character Recognition (STPOCR), is commonly used to improve the accuracy and efficiency of scene understanding and object recognition tasks. Specifically, the STPOCR algorithm is a recognition framework that combines spatial position prediction with parallel decoding. This is achieved through a discretized position encoding and an instance allocation decoder, enabling the simultaneous localization and recognition of multiple text instances.

[0067] After the ladle number detection model training is completed, the ladle number detection model is started to detect the real-time ladle number image to obtain the ladle number main body image; then the ladle number main body image is input into the trained text recognition model, and the ladle number main body image is recognized by the text recognition model to obtain the ladle number recognition result.

[0068] For the text recognition model, images of printed serial numbers are captured in specific industrial scenarios and annotated. Using an image annotation tool, a point and text information are annotated at the center of each character to create a text recognition dataset. The dataset is divided into three parts: a training set, a test set, and a validation set. The training set is used to train the end-to-end text recognition model. Specifically, the text recognition training set contains basic image attributes and annotation information. Basic image attributes include filename, width, height, and depth. Annotation information includes the x-coordinate of the text center, the y-coordinate of the text center, and the text content. The training dataset is generated by single-point annotation of the main image of the ladle number. The text recognition model is then trained using the end-to-end text scene recognition (STPOCR) algorithm. Finally, the trained text recognition model is deployed on the Nvidia Jetson platform, where it is activated to detect and output text content.

[0069] NVIDIA Jetson is an embedded system designed by NVIDIA for the next generation of autonomous machines. Each system is a complete System-on-Module (SOM) featuring a CPU, GPU, PMIC, DRAM, and flash memory, enabling a wide range of applications requiring a variety of performance levels and price points. Using onnxruntime, CPU or GPU versions can be deployed based on project requirements, bypassing the difficulties of directly converting STPOCR models to TensorRT deployment. This ensures that models can be converted from ONNX models to TensorRT models using trtexec on the Jetson NX development board, allowing TensorRT inference to be performed in C++. The average inference time for each ladle number image is less than 50ms.

[0070] This application processes images using a text recognition model, treating the entire OCR process as a whole and directly converting the original image into the final text result. The advantage of this approach is that it avoids the problems caused by staged processing and improves recognition accuracy and stability.

[0071] In one embodiment, the process of recognizing the main image of the ladle number through the text recognition model includes: preprocessing, sequence construction, instance allocation decoder (IAD), information transmission, parallel recognition decoder (PRD), reasoning and post-processing; through the design of instance allocation decoder (IAD) and parallel recognition decoder (PRD), the limitations of the maximum word text length and the maximum line text length are overcome.

[0072] See also Figure 2 , Figure 2This is a flow chart for identifying the main image of the ladle number in one embodiment of the present application. Figure 2 In the example, the trained text recognition model is used to identify the main image of the ladle number, including:

[0073] Step S210, extracting features from the main image of the ladle number through a feature extraction module in the text recognition model to obtain a continuous text sequence;

[0074] Step S220, converting the continuous text sequence into a discretized text sequence through a discretization module in the text recognition model, and identifying one or more text instances in the discretized text sequence;

[0075] Step S230, predicting the center point positions of one or more text instances by using an instance allocation decoder in the text recognition model to obtain the center point positions of the one or more text instances;

[0076] Step S240 , using the parallel recognition decoder in the text recognition model to predict in parallel the ladle number corresponding to each text instance based on the text instance and the center point position of the text instance.

[0077] Specifically, the ladle number main body image is preprocessed by the feature extraction module to form a continuous text sequence containing global semantics. This text sequence is then segmented into multiple independent text instances by the discrete module to complete the sequence construction. The instance assignment decoder automatically predicts the center point position of each text instance by analyzing the geometric features of the text instance. The parallel recognition decoder simultaneously decodes the features of multiple text instances based on the text instance and the center point position of the text instance, predicting the ladle number corresponding to each text instance, thus avoiding the traditional method's reliance on character arrangement order. The entire processing flow achieves automated processing from image input to multi-instance recognition through an end-to-end model architecture, effectively overcoming the technical obstacles of manual labeling, which is inefficient and difficult to adapt to complex arrangements.

[0078] More specifically, continuous text instances are converted into discrete sequences, which can be converted into sequences through a simplified single-point representation. The instance allocation decoder (IAD) autoregressively predicts the center point coordinates (x and y) of each text instance in the sequence until the end of the sequence. For example, an image with a side length of 800 pixels only needs n bins = 800 to achieve zero quantization error. The center point of a text instance is obtained by averaging the midpoints of the upper and lower edges. A text instance can be represented by a three-part sequence, [x, y, t], where (x, y) are discrete coordinates and t is the transcript to be predicted by the parallel recognition decoder (PRD). The transcript is discrete, meaning each character represents a category.

[0079] It should be noted that the feature extraction module can be implemented using a convolutional neural network combined with a Transformer encoder to capture the continuous arrangement features and contextual features of the ladle number characters.

[0080] See also Figure 3 , Figure 3 This is a flow chart of a method for obtaining the center point position of a text instance according to an embodiment of the present application. Figure 3 In , the methods for obtaining the center point position of a text instance include:

[0081] Step S310: inserting the first tag into the head of the text instance and inserting the second tag into the tail of the text instance to obtain a first text instance;

[0082] Step S330, detecting the coordinates (xi, yi) of the center point of the first text instance;

[0083] Step S330: discretize the coordinates of the center point of the text instance into [1, n bins ], where n bins Control the degree of dispersion;

[0084] Step S340: Use the discretized coordinates to represent the position features of the text instance to obtain the center point position of the text instance.

[0085] Among them, the first marker is a symbol used to identify the starting position of the text instance, which can be implemented by special characters or preset codes, such as the marker SOS, which is used to clarify the starting boundary of the instance by inserting the position. The second marker refers to the symbol used to identify the ending position of the text instance, which can be implemented by using a symbol or code different from the first marker, such as the marker EOS, which is used to define the ending boundary of the instance by inserting the position. If, given an image containing N text instances, the constructed sequence will include 2N discrete markers, in which the text instances will be randomly sorted. In this way, randomly sorted text instances can be effectively learned, thereby realizing implicit label assignment, avoiding the same explicit label assignment as using the bisection method, and reducing the amount of computation for empty text examples.

[0086] Markers are inserted at the beginning and end of a text instance so that the model can determine the physical range of the instance based on the marker position and effectively distinguish adjacent or overlapping text instances. By detecting the coordinates of the center point (xi, yi) between the markers, the geometric center of the text instance can be located. The continuous coordinates are then discretized to a fixed interval [1, n bins ] so that the model does not need to process high-precision floating-point operations. The discrete degree parameter n binsThe introduction of allows the discretization granularity to be adjusted according to the actual scenario, for example, increasing the discrete interval in scenarios requiring high-precision positioning and reducing the discrete interval in scenarios with limited computing resources. The discretized integer coordinates are input into the model as position features.

[0087] In one embodiment, a method for obtaining a continuous text sequence includes: extracting features of a main image of a ladle number through a convolutional neural network to obtain a first feature map; and inputting the first feature map into a Transformer encoder to obtain a text sequence.

[0088] Among them, convolutional neural networks, deep neural networks with spatial feature extraction capabilities, can be implemented using structures such as CNN (Convolutional Neural Networks, CNN), and are used to extract multi-level visual information containing local detail features from ladle number images.

[0089] Among them, the Transformer encoder, a feature processing module based on the self-attention mechanism, can be implemented using a stacked structure consisting of a multi-head attention layer and a feedforward network layer, which is used to perform global context modeling on the feature map output by the convolutional neural network.

[0090] Specifically, the main image of the ladle number is first subjected to feature extraction by a convolutional neural network. The convolution layer gradually extracts spatial features such as character edges and stroke directions in the image through the local receptive field, generating a first feature map containing spatial dimension information. The first feature map is then input into the Transformer encoder, and the association weights between features at different positions are calculated through the self-attention mechanism, so that nonlinearly arranged characters can establish position correlation based on the global context. The instance assignment decoder predicts the center point position of one or more text instances to obtain the center point position of one or more text instances;

[0091] The instance allocation decoder IAD and the parallel recognition decoder PRD share the same Transformer decoder parameters and are jointly supervised by the detection and recognition gradients, ensuring the simplicity of the algorithm flow.

[0092] In one embodiment, the loss function during text recognition model training is:

[0093]

[0094] Where L is the sequence length, w i is the likelihood weight of the i-th mark, For input S 1:i Under the condition that the input image I has the output sequence The probability of S is the input sequence. It should be noted that w iBased on experience, it can be set to 1.

[0095] In one embodiment, before the step of parallelly predicting the ladle number corresponding to each text instance based on the text instance and the center point position of the text instance through the parallel recognition decoder in the text recognition model, the method also includes: extracting hidden text instance position features and corresponding text position prediction results; converting the text position prediction results into embedded data; and adding the embedded data to the text instance position features.

[0096] Among them, the hidden text instance position feature is used to represent the potential position relationship of the text instance in the image, which can be implemented using a convolutional layer or an attention mechanism.

[0097] Among them, embedded data, the process of mapping numerical position prediction results into vector form, can be achieved through a fully connected layer or an embedding matrix, so that the position information can be aligned with the feature space of the deep learning model.

[0098] Among them, adding embedded data to the text instance position feature can be achieved by element-by-element addition or channel splicing.

[0099] Extract the hidden text instance position features and text position prediction results of the Instance Assignment Decoder (IAD), then convert the text position prediction results into embedded data, and combine these embedded data with the text position features for use in the subsequent parallel recognition decoder. The specific process is as follows:

[0100]

[0101] With the help of information transfer, the gradient of text recognition can be or Passed to different text instances, in the Parallel Recognition Decoder (PRD) stage, the recognition tag can perceive the previously detected text information through features and use this prior information as the first two queries to instruct the decoder to recognize all text instances in parallel.

[0102] By extracting hidden text instance position features, the relative position and arrangement of text instances in the image can be captured. The text position prediction results are converted into embedded data, and the converted embedded data is fused with the position features to form enhanced features containing precise position information. This processing enables the parallel recognition decoder to simultaneously utilize the semantic features of the text content and the enhanced position features when predicting the ladle number.

[0103] In one embodiment, during the process of predicting the ladle number corresponding to each text instance in parallel, the method further includes: padding or truncating the text instance to a fixed length K, and adding a gap marker to fill the gap of the short text instance.

[0104] The fixed length K refers to a pre-set uniform dimension parameter, which can be determined empirically or through experimental optimization. For example, K can be set to the maximum expected number of characters plus the number of buffer bits to achieve dimensional alignment of the input tensors during batch processing. The vacancy marker is a special symbol used to identify vacant positions. This can be implemented using a uniformly encoded placeholder, such as a zero vector or a preset character code, to prevent invalid positions from interfering with the model's attention mechanism.

[0105] Specifically, during the parallel processing phase, the feature sequences of all text instances are forced to be of the same length, K. For text instances longer than K, a truncation operation is used to retain the first K valid characters; for text instances less than K, a padding operation is used to add missing tokens until K is reached. The missing tokens can be PDAs. Thus, text instances of different lengths are converted into matrix inputs of uniform dimension, eliminating the batch processing obstacles caused by differences in sequence length. The introduction of missing tokens enables the model to distinguish between valid characters and padding areas, preventing redundant positions from participating in weight calculations, thereby reducing computing resource consumption.

[0106] For inference, the center point coordinates provided by the Instance Assignment Decoder (IAD) are used to predict the recognition results of each text instance in parallel. These predictions are made simultaneously, significantly reducing the sequence length and cycle length required for inference.

[0107] For the cycle length, if n cls character categories, the vocabulary size of the dictionary used to mark the sequence can be counted as n cls +3, where the additional three categories are the first token SOS at the head of the text instance by the instance assignment decoder (IAD), the second token EOS at the tail of the text instance, and the vacant token PAD used to fill the short text instance. According to experience, K and n are bins are set to 25 and 1000 respectively. In addition, the maximum length of the text instance n is set to 25 and 1000 respectively. ti Set to 60. An image contains N text instances, each containing up to K characters. STPOCR needs N v cycles, where N v Defined as:

[0108] N v =2N+K+1

[0109] For post-processing, for the prediction results output by the text recognition model, the confidence score is calculated by calculating the average likelihood of all tags in each paragraph, and then a threshold is set based on this score to filter the original output, effectively removing redundant and false positive predictions.

[0110] This application uses single-point annotation for both linear and nonlinear text, which improves annotation efficiency. It uses an end-to-end transformer architecture, and the character recognition accuracy can reach over 99%. It uses fewer parameter training to reduce training time and achieves faster inference speed. The average inference time for each ladle number image is within 50ms.

[0111] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0112] Figure 4 This is a block diagram of a ladle number recognition device according to an embodiment of the present application. Figure 4 As shown, a ladle number identification device includes:

[0113] Image detection module 410, for detecting the main image of the ladle number; wherein the main image of the ladle number is obtained by detecting the ladle number detection model that has been trained;

[0114] The image recognition module 420 is used to use the trained text recognition model to recognize the main image of the ladle number, and use the text content output by the text recognition model as the ladle number recognition result; wherein, the training sample for training the text recognition model is obtained by single-point annotation of the main image of the ladle number, and the text recognition model is obtained based on the end-to-end text scene recognition STPOCR algorithm.

[0115] It should be noted that the ladle number identification device provided in the above embodiment and the ladle number identification method provided in the above embodiment are based on the same concept, wherein the specific manner in which each module and unit performs operations has been described in detail in the method embodiment and will not be repeated here. In actual applications, the ladle number identification device provided in the above embodiment can allocate the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0116] An embodiment of the present application also provides a ladle number identification device, including: one or more processors; and a memory for storing one or more programs, when the one or more programs are executed by one or more processors, the memory implements the ladle number identification method in the above embodiment.

[0117] The embodiments of the present application further provide one or more machine-readable media having instructions stored thereon, which, when executed by one or more processors, enable the processors to execute the ladle number identification method in the above-mentioned embodiment.

[0118] Figure 5FIG1 shows a schematic diagram of a computer system structure suitable for implementing a memory according to an embodiment of the present invention. It should be noted that Figure 5 The computer system of the memory shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0119] like Figure 5 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage portion into the random access memory (RAM) 503, such as executing the method in the above embodiment. Various programs and data required for system operation are also stored in the RAM. The CPU 501, ROM 502, and RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0120] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, and the like; an output section 507 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 508 including a hard disk; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card or a modem. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. Removable media 511, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 510 as needed, so that computer programs read therefrom can be installed into the storage section 508 as needed.

[0121] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for executing the ladle number identification method described above. In such an embodiment, the computer program can be downloaded and installed from a network via a communication component and / or installed from removable media 511. When executed by the central processing unit (CPU) 501, the computer program performs various functions defined in the system of the present invention.

[0122] It should be noted that the computer-readable medium shown in the embodiments of the present invention may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. The computer-readable storage medium may, for example, be an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM) 503, a read-only memory (ROM) 502, an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable computer program. Such a propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. A computer program embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0123] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. Each box in the flowchart or block diagram can represent a module, program segment, or part of the code, and the above-mentioned module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the boxes can also occur in an order different from that marked in the accompanying drawings. For example, two boxes shown in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0124] The units involved in the embodiments of the present invention may be implemented in software or hardware, and the units described may also be provided in a processor. In some cases, the names of these units do not limit the units themselves.

[0125] Another aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon. When executed by a computer processor, the computer program causes the computer to perform the aforementioned ladle number identification method. The computer-readable storage medium may be included in the memory described in the above embodiments, or may exist independently and not be incorporated into the memory.

[0126] Another aspect of the present invention provides a computer program product or computer program, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the ladle number identification method provided in each of the above embodiments.

[0127] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, any equivalent modifications or alterations accomplished by a person of ordinary skill in the art without departing from the spirit and technical concepts disclosed in this application shall be covered by the claims of this application.

Claims

1. A ladle number identification method, characterized in that: The ladle number identification method comprises: Detecting the main image of the ladle number; wherein the main image of the ladle number is obtained by detecting the trained ladle number detection model; The ladle number main body image is recognized using a trained text recognition model, and the text content output by the text recognition model is used as the ladle number recognition result; wherein, the training samples for training the text recognition model are obtained by single-point annotation of the ladle number main body image, and the text recognition model is obtained based on the end-to-end text scene recognition STPOCR algorithm.

2. The ladle number identification method according to claim 1, characterized in that: The method of using the trained text recognition model to identify the main image of the ladle number includes: Performing feature extraction on the main image of the ladle number by using the feature extraction module in the text recognition model to obtain a continuous text sequence; Converting the continuous text sequence into a discretized text sequence through a discretization module in the text recognition model, and identifying one or more text instances in the discretized text sequence; Predicting the center point positions of the one or more text instances by using an instance allocation decoder in the text recognition model to obtain the center point positions of the one or more text instances; The parallel recognition decoder in the text recognition model predicts the ladle number corresponding to each text instance in parallel according to the text instance and the center point position of the text instance.

3. The ladle number identification method according to claim 2, characterized in that: The method for obtaining the center point position of the text instance includes: Inserting the first tag into the head of the text instance and inserting the second tag into the tail of the text instance to obtain a first text instance; Detecting the coordinates (xi, yi) of the center point of the first text instance; The coordinates of the center point of the text instance are discretized into [1, n bins ], where n bins Control the degree of dispersion; The discretized coordinates are used to represent the position features of the text instance and the center point position of the text instance is obtained.

4. The ladle number identification method according to claim 2, characterized in that: The method for obtaining the continuous text sequence includes: Performing feature extraction on the main image of the ladle number through a convolutional neural network to obtain a first feature map; The first feature map is input into the Transformer encoder to obtain a text sequence.

5. The ladle number identification method according to claim 2, characterized in that: The loss function during the text recognition model training process is: Where L is the sequence length, w i is the likelihood weight of the i-th mark, For input S 1:i Under the condition that the input image I has the output sequence The probability of , S is the input sequence.

6. The ladle identification method according to claim 2, characterized in that: Before the step of predicting in parallel the ladle number corresponding to each text instance based on the text instance and the center point position of the text instance by the parallel recognition decoder in the text recognition model, the method further includes: Extract hidden text instance position features and corresponding text position prediction results; Converting the text position prediction result into embedded data; The embedded data is added to the text instance position feature.

7. The ladle identification method according to claim 2, characterized in that: In the process of predicting the ladle number corresponding to each text instance in parallel, the method further includes: The text instances are padded or truncated to a fixed length K, and a gap marker is added to fill the gaps of short text instances.

8. A ladle number identification device, characterized in that: The ladle number identification device comprises: An image detection module is used to detect the main image of the ladle number; wherein the main image of the ladle number is obtained by detecting the trained ladle number detection model; An image recognition module is used to identify the main image of the ladle number using a trained text recognition model, and use the text content output by the text recognition model as the ladle number recognition result; wherein, the training samples for training the text recognition model are obtained by single-point annotation of the main image of the ladle number, and the text recognition model is obtained based on the end-to-end text scene recognition STPOCR algorithm.

9. A ladle number identification device, characterized in that: include: one or more processors; and A memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the memory implements the ladle number identification method according to any one of claims 1 to 7.

10. A machine-readable medium, characterized in that Instructions are stored thereon, which, when executed by one or more processors, enable the processors to execute the ladle number identification method according to any one of claims 1 to 7.

Images