Method, equipment and computer program product for automatically metering plates

Abstract

The invention discloses a method and device for automatically metering plates and a computer program product, and the method comprises the steps: detecting whether a platform scale used for metering the plates is empty or not; under the condition that the platform scale is not empty, identifying the spraying mark of the plate on the platform scale to obtain an identification result; detecting whether the identification result is correct or not; and under the condition that the identification result is correct, controlling the platform scale to meter the plate. According to the technical scheme provided by the invention, the plates on the platform scale can be automatically and accurately metered, and the intelligent management level of a production line is improved.

Description

Technical Field

[0001] The present application belongs to the field of image recognition technology, and in particular, relates to a method, device and computer program product for automatic plate measurement. Background Art

[0002] In today's steel manufacturing industry, trade scales are often used for weighing finished plate products. The scales are scattered and the operation process is mostly done manually, making it impossible to achieve centralized control. At the same time, depending on the different on-site businesses, there are many business scenarios such as single-plate measurement, multi-plate measurement, and group plate measurement with different measurement orders.

[0003] In the case of continuous production of plates, scales and track scales are used very frequently, and the continuous operation intensity of personnel is high. In order to solve the above production business pain points, it is urgent to build a complete automatic measurement system to achieve reduction in staff and increase efficiency while improving the intelligent management level of the production line. Summary of the invention

[0004] The embodiments of the present application provide a method, device and computer program product for automatic plate measurement, which can automatically measure the plates at least to a certain extent and improve the intelligent management level of the production line.

[0005] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0006] According to a first aspect of an embodiment of the present application, a method for automatic plate measurement is provided, comprising:

[0007] Check whether the scale used to weigh the plates is empty;

[0008] When the scale is not empty, identifying the spray mark of the plate on the scale to obtain an identification result;

[0009] Detecting whether the recognition result is correct;

[0010] When the recognition result is correct, the scale is controlled to weigh the plate.

[0011] In some embodiments, a preset mark is provided on the scale, and the step of detecting whether the scale used for weighing the plate is empty includes:

[0012] When the cross-body vehicle loaded with plates reaches the preset position, an image of the scale is obtained;

[0013] Determining whether the preset mark exists in the scale image;

[0014] When the preset mark exists in the scale image, it is determined that the scale for weighing the plate is not empty.

[0015] In some embodiments, determining whether the preset mark exists in the scale image includes:

[0016] Converting the scale image into a binary image;

[0017] Determining a connected area in the binary image;

[0018] According to the size of the connected area and the size of the preset mark on the scale, it is determined whether the preset mark exists in the scale image.

[0019] In some embodiments, the identifying the spray mark of the plate on the scale to obtain the identification result includes:

[0020] Acquire a spray marking image of the plate on the scale;

[0021] The spray mark image is sequentially subjected to angle rotation, Gaussian filtering, top hat transformation, binarization, region of interest positioning and segmentation processing, and then input into a preset neural network model to obtain a recognition result of the spray mark output by the preset neural network model.

[0022] In some embodiments, the preset neural network model includes, in sequence: a ResNet-50 network, a slice grouping network, a fast convolutional neural network, a character segmentation and a spatial attention module.

[0023] In some embodiments, the method for automatic plate measurement further includes:

[0024] Performing text block division and text block sorting on the recognition results to obtain sorted text blocks;

[0025] The spraying information is obtained according to the preset spraying identification format and the sorted text blocks.

[0026] In some embodiments, the detecting whether the recognition result is correct includes:

[0027] When the spraying information exists in the preset database, it is determined that the recognition result is correct.

[0028] In some embodiments, the method for automatic plate measurement further includes:

[0029] Communicate with the image capture device based on socket communication protocol and message queue.

[0030] According to a second aspect of an embodiment of the present application, there is provided a device for automatic plate measurement, comprising a processor and a memory, wherein the memory stores computer program instructions that can be executed by the processor, and when the processor executes the computer program instructions, the steps of the method described in any one of the first aspects above are implemented.

[0031] According to a third aspect of an embodiment of the present application, a computer program product is provided, including a computer program. When the computer program product is executed by a processor, it prompts the processor to implement the steps of the method described in any one of the first aspects above.

[0032] In the present application, it is detected whether the scale used for weighing the plates is empty; if the scale is not empty, the spray mark of the plate on the scale is identified to obtain an identification result; it is detected whether the identification result is correct; if the identification result is correct, the scale is controlled to weigh the plate. The technical solution provided by the present application can automatically and accurately weigh the plates on the scale, thereby improving the intelligent management level of the production line.

[0033] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0035] Figure 1 A schematic diagram of a process flow of a method for automatic plate measurement in one embodiment is shown;

[0036] Figure 2 A schematic diagram of an image of an empty scale in one embodiment is shown;

[0037] Figure 3 A schematic diagram showing a spray-painted marking image of a plate in one embodiment is shown;

[0038] Figure 4 A schematic diagram showing an image of a scale with a preset mark in one embodiment is shown;

[0039] Figure 5 A schematic diagram showing a connection area in one embodiment is shown;

[0040] Figure 6 A flowchart of processing a spray marking image of a plate in one embodiment is shown;

[0041] Figure 7 A processing flow chart of a preset neural network model in one embodiment is shown;

[0042] Figure 8 A schematic diagram of the structure of a ResNet-50 network in one embodiment is shown;

[0043] Fig. 9 A schematic diagram showing a recognition result in one embodiment is shown;

[0044] Fig.10 A schematic diagram showing spraying information in one embodiment is shown;

[0045] Fig.11 A schematic structural diagram of a device for automatic plate measurement in one embodiment is shown. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0047] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0048] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0049] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0050] Figure 1FIG. 2 is a flow chart of a method for automatic plate measurement in one embodiment. Figure 1 As shown, the method for automatic plate measurement may include the following steps 101 to 104 .

[0051] In step 101, it is detected whether a scale for weighing plates is empty.

[0052] It should be noted that, during the weighing (hereinafter referred to as measurement) of plates (such as steel plates), it is necessary to know whether the scale used to weigh the plates is empty, as well as the information of the plates currently being measured, which can be obtained from the inkjet code of the plates. Based on this, during weighing, an image of the scale can be captured by an image capture device (such as Figure 2 As shown) and the plate's inkjet identification image (as shown Figure 3 As shown), the image recognition technology is then used to determine whether the scale is empty and obtain the information of the plate.

[0053] In the actual implementation process, when plate measurement is required, the plates will be loaded onto the cross-carriage. After passing the rangefinder and the infrared grille, the cross-carriage will stay in a steady state. At this time, the cross-carriage will send a signal to the image capture device that it has reached the preset position. The image capture device calls the program SDK component to complete the cruise capture of the scale image, and triggers the detection of whether the scale is empty through telegram interaction.

[0054] By analyzing the captured scale image, it is found that the image capture device does not capture the inkjet code mark of the plate, so it is not possible to determine whether the scale is empty by detecting whether there is an inkjet code mark. Based on this, the embodiment of the present application sets a preset mark on the scale, and when the cross-car loaded with the plate arrives at the preset position, the scale image is obtained; it is determined whether there is a preset mark in the scale image; when the preset mark is in the scale image, it is determined that the scale used to weigh the plate is not empty.

[0055] Figure 4 FIG. 1 is a schematic diagram showing an image of a scale with a preset mark in one embodiment. Figure 4 As shown, in order to facilitate identification of the preset mark, the color of the preset mark is quite different from that of the scale. In order not to affect the accuracy of the measurement data, the weight of the preset mark should be lighter. In some embodiments, the preset mark can be a printed QR code, which is significantly different from the color of the scale and can be identified by a binarization method, and does not affect the measurement accuracy of the plate.

[0056] In some embodiments, the scale image can be converted into a binary image; a connected area can be determined in the binary image; and whether a preset mark exists in the scale image can be determined based on the size of the connected area and the size of the preset mark on the scale.

[0057] In the implementation process, the scale image can be first converted into a grayscale image, and then the grayscale image can be binarized. By combining adaptive binarization and binarization with a given threshold, the bright part is white and the rest is black to obtain a binary image. By filling the holes in the binary image, the area with the preset mark is filled with white to obtain a connected area. According to the size of the preset mark on the scale, the size of the preset mark in the scale image can be calculated, and then the size of the connected area is compared with the size of the preset mark in the scale image. If the two are close, it can be determined that the preset mark is stored in the scale image.

[0058] By using a binarization method to determine whether a scale is empty, it is possible to determine with high precision whether a preset mark exists in the scale image, thereby improving the accuracy of empty scale determination.

[0059] In some embodiments, the device for automatic plate weighing can communicate with the image capturing device and the scale based on a socket communication protocol and a message queue.

[0060] It can be understood that by communicating with the image capture device based on the socket communication protocol, a combination of long connection and short connection can be achieved, which minimizes the occupation of network resources while ensuring stable communication.

[0061] For special scenarios with multi-process and high concurrency of image library traversal and identification requests, we can effectively interact with image capture devices and scales based on the communication method of message queues to ensure that the telegram services with logical sequence can be executed in order. By establishing a storage and backtracking mechanism for communication telegrams, it is easy to find problems.

[0062] In some embodiments, when the scale is empty, the plate automatic weighing device controls the scale to perform a tare weighing operation. When the scale is not empty, the plate automatic weighing device needs to identify the spray mark of the plate on the scale.

[0063] In step 102, when the scale is not empty, the spray mark of the plate on the scale is identified to obtain an identification result.

[0064] The recognition of spray-painted markings on plates can be achieved using optical character recognition (OCR) technology. The general process of traditional OCR algorithms is to first pre-process the spray-painted marking image of the plate on the scale to make the character area features more obvious, then locate the character area based on the image features and perform correction and character segmentation, and finally input it into the classifier to obtain character information. For specific simple scenes, traditional OCR algorithms can recognize text information very well, but for complex and changeable scenes, traditional OCR algorithms are difficult to locate and segment in a variety of scenes. Figure 3 As can be seen from the example images, the spray-painted logo images taken during the plate metering process have the characteristics of unstable text angles, uneven lighting, complex and changeable character area formats and backgrounds. Traditional OCR algorithms are not suitable for the recognition of spray-painted logo images.

[0065] At present, OCR technology based on deep learning performs well in natural scene text recognition and can adapt to changing field conditions. OCR models based on deep learning can be divided into two types. The first is a two-stage model, which includes two models: text detection and text recognition. The text area is detected first, and then the recognition model is input to obtain text information; the second is an end-to-end model, that is, the image is input into a model to directly complete the detection and recognition of the text. Through the research and comparison of various models, the embodiment of the present application adopts an end-to-end neural network model to recognize spray-painted logo images.

[0066] In some embodiments, an image of a spray mark of a plate on a scale can be obtained; the spray mark image is subjected to angle rotation, Gaussian filtering, top hat transformation, binarization, region of interest positioning and segmentation processing in sequence, and then input into a preset neural network model to obtain a recognition result of the spray mark output by the preset neural network model.

[0067] Reference Figure 3 The first spray mark image is taken. Considering that the characters in some spray mark images are in the opposite direction, for a fixed scale, the direction of the characters photographed is basically the same. Therefore, before identifying the spray mark, a unified rotation angle can be set for the spray mark image of each plate on the scale. After the automatic plate weighing device reads the spray mark image, it rotates the spray mark image according to the rotation angle.

[0068] Reference Figure 3 The third spray logo image, considering that some spray logo images are blurred, needs to be processed by Gaussian filtering and other technical processing.

[0069] Figure 6 FIG. 1 shows a flow chart of processing a spray-painted marking image of a plate in one embodiment. Figure 6 As shown, after the spray mark image is rotated and Gaussian filtered, it can also be subjected to top-hat transformation, binarization, region of interest (ROI) positioning and segmentation processing, and then input into the preset neural network model to obtain the recognition result.

[0070] The preset neural network models include: ResNet-50 network, slice grouping network, fast convolutional neural network, character segmentation and spatial attention module.

[0071] It is understandable that the difficulty of scene text recognition lies in the fact that the text in the captured image has variable directions, lengths and shapes, which brings difficulties to text detection and recognition. At present, the detection and recognition effect of horizontal text in the field of OCR technology is better, but because most models use algorithms based on target detection for text detection, the effect is not good for text of arbitrary directions and shapes, while the segmentation algorithm can better complete this recognition task.

[0072] On the other hand, most of the current scene text detection algorithms for arbitrary shapes use a Region Proposal Network (RPN) to generate candidate boxes, but RPN relies on manually designed anchor points and has poor detection effects on text with extreme aspect ratios. In addition, the candidate boxes generated by RPN are represented by axis-aligned rectangles. For dense and tilted text, a single candidate box usually contains adjacent text, causing the ROI to contain more than just a single line of text features, affecting the text recognition effect. The model used in the embodiment of the present application uses a Segmentation Proposal Network (SPN) to generate candidate boxes, which can solve the limitations brought by RPN and is suitable for relatively dense inkjet characters on plates.

[0073] Figure 7 FIG. 1 shows a processing flow chart of a preset neural network model in one embodiment. Figure 7 As shown in the figure, the spray logo image is passed through the ResNet-50 network to obtain a fused feature map, and then the SPN network is used to generate candidate boxes. The fast convolutional neural network Fast R-CNN is used to fine-tune the candidate boxes. Finally, the character segmentation and spatial attention modules are used to recognize the text sequence to obtain the recognition result.

[0074] Figure 8 FIG. 4 shows a schematic diagram of the structure of a ResNet-50 network in one embodiment. Figure 8 As shown in the figure, when the spray logo image input to the ResNet-50 network is a single-channel binary image of size 64*40, the hidden layer of the ResNet-50 network contains 4 convolutional layers, 3 pooling layers, 2 fully connected layers and 2 Dropout layers, and the convolution kernel size of each convolutional layer is 3*3. The number of convolution kernels is 32, 64, 128, and 128 respectively; the size of the pooling units of the three pooling layers is 2*2. By adding a Dropout layer between the third and fourth convolutional layers and after the first fully connected layer, overfitting can be prevented. Figure 8 In the example, the length of the first fully connected layer is 5120, the length of the second fully connected layer is 1024, and the final output is 41.

[0075] It should be noted that in order to ensure the accuracy of the recognition results output by the neural network model, the spraying information of all plates can be collected, analyzed and classified, and stored in the database. Before the neural network model outputs the recognition results, the recognition results are verified in the following two aspects:

[0076] First, verify whether the recognition result meets the naming rules. For example, the last few digits of the furnace number should be the year, month, and day according to the naming rules. If the furnace number in the recognition result includes the letter "O", the letter "O" will be corrected to the number 0 before output.

[0077] Secondly, verify whether the recognition result exists in the database. If so, increase the output confidence. If not, conduct a comparative analysis of similar fields, supplemented by spray marking time limit, contract information, etc., and select the result closest to the business logic for output.

[0078] Fig. 9 FIG. 2 shows a schematic diagram of a recognition result in one embodiment. Fig. 9 As shown in the figure, the recognition results output by the neural network model may be slightly disordered and have no corresponding field information (such as plate number, furnace number, steel type, etc.). In order to more accurately determine the spraying information (such as Fig.10 As shown), the recognition results can be divided into text blocks and sorted to obtain sorted text blocks; according to the preset inkjet marking format and the sorted text blocks, the spraying information is obtained.

[0079] In the implementation process, the text blocks can be divided into rows by calculating the rotation angle and the y-axis coordinate of the center point of each text block in the recognition result, and then the order of the text blocks in each row is determined according to the x-axis coordinate of the center point to obtain the sorted text blocks. Since the inkjet mark has a fixed format, the sorted text blocks can be matched with the preset inkjet mark format to obtain the spraying information.

[0080] In step 103, it is detected whether the recognition result is correct.

[0081] In some embodiments, the recognition result can be determined to be correct if the recognition result exists in the preset database. If the recognition result is further analyzed to obtain the spraying information, the recognition result can be determined to be correct if the spraying information exists in the preset database.

[0082] In step 104, when the recognition result is correct, the scale is controlled to weigh the plate.

[0083] It is understandable that when the identification result is correct, the scale performs a gross measurement operation on the plate, and then the weight of the plate can be obtained based on the gross measurement result and the skin measurement result.

[0084] In some embodiments, when the recognition result is correct, the original spray-painted mark image and the recognition result can also be obtained, and the corresponding position on the original spray-painted mark image is marked, and then input as a sample into the neural network model for training to improve the stability of the neural network model.

[0085] In the above-mentioned method for automatic plate weighing, it is detected whether the scale used for weighing the plate is empty; if the scale is not empty, the spray mark of the plate on the scale is identified to obtain an identification result; it is detected whether the identification result is correct; if the identification result is correct, the scale is controlled to weigh the plate. The technical solution provided by this application can automatically and accurately weigh the plates on the scale, thereby improving the intelligent management level of the production line.

[0086] Based on the same inventive concept, the present application also provides a device for automatic plate measurement, referring to Fig.11 , shows a schematic diagram of the structure of the device for automatic plate measurement in an embodiment of the present application, wherein the device for automatic plate measurement includes one or more memories 1104, one or more processors 1102, and at least one computer program (computer program instruction) stored in the memory 1104 and executable on the processor 1102, and the processor 1102 implements the method described above when executing the computer program.

[0087] Among them, Fig.11 In the embodiment of the present invention, a bus architecture (represented by bus 1100) is shown, and bus 1100 may include any number of interconnected buses and bridges, and bus 1100 links various circuits including one or more processors represented by processor 1102 and memory represented by memory 1104. Bus 1100 may also link various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. Bus interface 1105 provides an interface between bus 1100 and receiver 1101 and transmitter 1103. Receiver 1101 and transmitter 1103 may be the same element, namely a transceiver, providing a unit for communicating with various other devices over a transmission medium. Processor 1102 is responsible for managing bus 1100 and general processing, while memory 1104 may be used to store data used by processor 1102 when performing operations.

[0088] Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium, in which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor is prompted to implement the steps of the method as described above.

[0089] Based on the same inventive concept, an embodiment of the present application provides a computer program product, including a computer program. When the computer program product is executed by a processor, it prompts the processor to implement the steps of the method as described above.

[0090] The functions described herein may be implemented in hardware, software executed by a processor, firmware, or any combination thereof. If implemented in software executed by a processor, the functions may be stored as one or more instructions or codes on a computer-readable medium or transmitted via a computer-readable medium. Other examples and implementations are within the scope and spirit of the present application and the appended claims. For example, due to the nature of software, the functions described above may be implemented using software executed by a processor, hardware, firmware, hard wiring, or a combination of any of these. In addition, each functional unit may be integrated into a processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit.

[0091] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0092] The units described as separate components may or may not be physically separated, and the components of the control device may or may not be physical units, that is, they may be located in one place or distributed in multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0093] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly or all or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc., which can store computer program instructions.

[0094] The above description is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of the claims of the present application.

Claims

1. A method for automatic plate measurement, characterized in that: include: Check whether the scale used to weigh the plates is empty; When the scale is not empty, identifying the spray mark of the plate on the scale to obtain an identification result; Detecting whether the recognition result is correct; When the recognition result is correct, the scale is controlled to weigh the plate.

2. The method for automatic plate measurement according to claim 1, characterized in that: The scale is provided with a preset mark, and the method of detecting whether the scale used for weighing the plate is empty includes: When the cross-body vehicle loaded with plates reaches the preset position, an image of the scale is obtained; Determining whether the preset mark exists in the scale image; When the preset mark exists in the scale image, it is determined that the scale for weighing the plate is not empty.

3. The method for automatic plate measurement according to claim 2, characterized in that: The determining whether the preset mark exists in the scale image includes: Converting the scale image into a binary image; Determining a connected area in the binary image; According to the size of the connected area and the size of the preset mark on the scale, it is determined whether the preset mark exists in the scale image.

4. The method for automatic plate measurement according to claim 1, characterized in that: The step of identifying the spray mark of the plate on the scale to obtain an identification result includes: Acquire a spray marking image of the plate on the scale; The spray mark image is sequentially subjected to angle rotation, Gaussian filtering, top hat transformation, binarization, region of interest positioning and segmentation processing, and then input into a preset neural network model to obtain a recognition result of the spray mark output by the preset neural network model.

5. The method for automatic plate measurement according to claim 4, characterized in that: The preset neural network model includes: ResNet-50 network, slice grouping network, fast convolutional neural network, character segmentation and spatial attention module.

6. The method for automatic plate measurement according to claim 5, characterized in that: Also includes: Performing text block division and text block sorting on the recognition results to obtain sorted text blocks; The spraying information is obtained according to the preset spraying identification format and the sorted text blocks.

7. The method for automatic plate measurement according to claim 6, characterized in that: The detecting whether the recognition result is correct includes: When the spraying information exists in the preset database, it is determined that the recognition result is correct.

8. The method for automatic plate measurement according to any one of claims 1 to 7, characterized in that: Also includes: Communicate with the image capture device based on socket communication protocol and message queue.

9. A device for automatic plate measurement, comprising a processor and a memory, characterized in that: The memory stores computer program instructions that can be executed by the processor, and when the processor executes the computer program instructions, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program product is executed by a processor, the processor is prompted to implement the steps of the method according to any one of claims 1 to 8.

Images