Steel ladle identification method and system, electronic equipment and storage medium

Abstract

The invention relates to a steel ladle identification method and system, electronic equipment and a storage medium, and belongs to the technical field of steel ladle detection.The method comprises the steps that a thermal infrared image and a visible light image shot for a steel ladle passing path are collected; wherein the shooting time difference of the thermal infrared image and the visible light image is smaller than a preset time difference value; judging whether the steel ladle appears in the thermal infrared image or not; and if the steel ladle appears in the thermal infrared image, identifying the visible light image to obtain information recorded on the steel ladle in the visible light image. According to the method, after the appearance of the steel ladle is detected through the thermal infrared image, the information recorded on the steel ladle is identified through the visible light image, so that the accuracy of visible light identification can be improved, and meanwhile, the computing power demand of visible light image identification is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of ladle detection, and in particular to a ladle identification method, system, electronic equipment and storage medium. Background Art

[0002] In steel mills, ladle is a key container for storing high-temperature molten steel. Its usage record, positioning and status monitoring are crucial to maintaining the logistics of steel mills. Ladle number recognition technology is an important means to achieve these goals. Common ladle number recognition methods include visible light image recognition methods, which use camera equipment to obtain a video stream, and then perform ladle detection and ladle number recognition on each visible light image in the video stream one by one. Although this ladle number recognition method based on visible light images is effective, it may face the problem of insufficient server computing power when deploying camera equipment at multiple locations, affecting concurrent processing capabilities. Summary of the invention

[0003] In view of this, it is necessary to provide a ladle identification method, system, electronic device and storage medium to solve the problem of insufficient examples in the ladle number recognition method based on visible light images in the related art.

[0004] In order to solve the above problems, in a first aspect, the present invention provides a ladle identification method, comprising: Collecting a thermal infrared image and a visible light image taken of the path of the ladle; wherein the difference between the shooting time of the thermal infrared image and the visible light image is less than a preset time difference; determining whether the ladle appears in the thermal infrared image; If the steel ladle appears in the thermal infrared image, the visible light image is identified to obtain the information recorded on the steel ladle in the visible light image.

[0005] Optionally, the determining whether the ladle appears in the thermal infrared image comprises: Determine whether the ladle appears in the thermal infrared image, and if the ladle appears in the thermal infrared image, whether the position of the ladle in the thermal infrared image meets the preset position requirement; If the ladle appears in the thermal infrared image, identifying the visible light image includes: If the ladle appears in the thermal infrared image and the position of the ladle in the thermal infrared image meets the preset position requirement, the visible light image is identified.

[0006] Optionally, the determining whether the ladle appears in the thermal infrared image, and if the ladle appears in the thermal infrared image, whether the position of the ladle in the thermal infrared image meets a preset position requirement, includes: Identify the thermal infrared image by using a ladle recognition algorithm, and determine whether the ladle appears in the thermal infrared image, and if the ladle appears in the thermal infrared image, whether the position of the ladle in the thermal infrared image meets a preset position requirement; The ladle recognition algorithm is trained in the following way: Acquire a sample thermal infrared image taken for the path of the ladle, and a labeled thermal infrared image with the ladle area labeled; The ladle recognition algorithm is trained based on the sample thermal infrared image and the annotated thermal infrared image through a convolutional neural network.

[0007] Optionally, the information recorded on the ladle includes the ladle number; and the identifying the visible light image includes: The visible light image is recognized by an optical character recognition algorithm to obtain the ladle number recorded on the ladle in the visible light image.

[0008] Optionally, the method further comprises: If the ladle appears in the thermal infrared image and the position of the ladle in the thermal infrared image meets the preset position requirement, temperature information recognition is performed on the thermal infrared image to obtain the temperature information of the ladle.

[0009] Optionally, the temperature information includes a temperature matrix; and the method further includes: Determining the state of the ladle according to the temperature matrix and a preset temperature evaluation algorithm; When the state of the ladle does not meet the preset state requirement, an alarm message is generated.

[0010] Optionally, the information recorded on the ladle includes the ladle number; and the method further comprises: If the ladle appears in the thermal infrared image and the position of the ladle in the thermal infrared image meets the preset position requirement, the target information of the ladle is recorded and stored; the target information includes: the position of the camera equipment that takes the thermal infrared image and the visible light image, the visible light image, the thermal infrared image, the ladle number, the temperature matrix and at least one of the state of the ladle.

[0011] In a second aspect, the present invention further provides a ladle identification system, comprising: a camera device and a server, wherein the camera device and the server are communicatively connected; The camera device is used to: take a thermal infrared image and a visible light image of the ladle's route, and send the thermal infrared image and the visible light image to the server; wherein the difference between the shooting time of the thermal infrared image and the visible light image is less than a preset time difference; The server is used to: determine whether the ladle appears in the thermal infrared image; if the ladle appears in the thermal infrared image, identify the visible light image to obtain information recorded on the ladle in the visible light image.

[0012] In a third aspect, the present invention further provides an electronic device comprising a memory and a processor, wherein the memory is used to store programs; and the processor is coupled to the memory and is used to execute the programs stored in the memory to implement the steps in any one of the above-mentioned ladle identification methods.

[0013] In a fourth aspect, the present invention further provides a computer-readable storage medium for storing a computer-readable program, wherein the program or instruction, when executed by a processor, can implement the steps in any one of the above-mentioned ladle identification methods.

[0014] The beneficial effects of the present invention are: After obtaining the thermal infrared image and the visible light image taken for the route of the ladle, the shooting time difference is within the preset time difference, and then the thermal infrared image is first identified. If it is identified that there is no ladle in the thermal infrared image, no further identification operation is performed on the visible light image; if it is identified that there is a ladle in the thermal infrared image, the visible light image is further identified to obtain the information recorded on the ladle in the visible light image. On the one hand, compared with the visible light image, the thermal infrared image has a single color and a low image pixel, so the recognition model structure is simple, so that the recognition of whether the ladle appears or not through the thermal infrared image is compared with the recognition of whether the ladle appears or not through the visible light image, and the computing power requirements of the server are lower. On the other hand, because the ladle has a high temperature, it has obvious characteristics in the thermal infrared image, so even if the training material is less, the ladle recognition rate based on the thermal infrared image can reach a very high level. The ladle information detection method combining thermal infrared image and visible light image not only has higher recognition efficiency and recognition accuracy than the method of using a single visible light image for recognition, but also supports more concurrent detection requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic flow chart of an embodiment of a ladle identification method provided by the present invention; Figure 2 A schematic flow chart of another embodiment of the ladle identification method provided by the present invention; Figure 3A framework diagram of the ladle identification method provided by the present invention; Figure 4 A schematic structural diagram of an embodiment of a ladle identification system provided by the present invention. DETAILED DESCRIPTION

[0016] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0017] In the description of the embodiments of the present invention, unless otherwise specified, "multiple" means two or more than two. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" may mean: A exists alone, A and B exist at the same time, and B exists alone.

[0018] The descriptions of "first", "second", etc. involved in the embodiments of the present invention are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the technical features defined as "first" and "second" may explicitly or implicitly include at least one of the features.

[0019] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0020] Reference Figure 1 , shows a schematic flow chart of an embodiment of a ladle identification method provided by the present invention, the method comprising: Step S101, collecting thermal infrared images and visible light images of the path of the ladle; wherein the difference between the shooting time of the thermal infrared image and the visible light image is less than a preset time difference.

[0021] Step S102, determining whether the ladle appears in the thermal infrared image.

[0022] Step S103: If the ladle appears in the thermal infrared image, the visible light image is identified to obtain information recorded on the ladle in the visible light image.

[0023] The ladle is usually transported via rails. Therefore, if the use of the ladle needs to be monitored, a camera device may be installed near the rails, for example, at both ends of the rails for monitoring.

[0024] The camera device may be installed at one or more locations. The camera device may take thermal infrared images and visible light images of the ladle's route. The thermal infrared image and the visible light image may be taken by the same camera device or by different camera devices.

[0025] After obtaining the thermal infrared image and the visible light image taken for the ladle's passing route, the thermal infrared image and the visible light image whose shooting time difference is less than a preset time difference can be taken as a group for subsequent identification and detection.

[0026] For each group of images, the thermal infrared image in the group of images can be first identified, and it can be determined whether a ladle appears in the thermal infrared image. If no ladle appears in the thermal infrared image, the visible light image in the group is not identified; if a ladle appears in the thermal infrared image, the visible light image in the group is identified, and the information recorded on the ladle in the visible light image is identified.

[0027] In the embodiment of the present invention, the use of thermal infrared images to identify the presence of a ladle requires lower computing power on the server than the use of visible light images in the related art. At the same time, due to the high temperature of the ladle, it has obvious features in the thermal infrared image, so the ladle recognition rate based on the thermal infrared image can also reach a very high level. The ladle information detection method combining thermal infrared images and visible light images not only has higher recognition efficiency and accuracy than the method of using visible light images alone for recognition, but also supports more concurrent detection requirements.

[0028] In one embodiment, the information recorded on the ladle includes the ladle number; then step S101 may specifically include: recognizing the visible light image by an optical character recognition algorithm to obtain the ladle number recorded on the ladle in the visible light image.

[0029] In one embodiment, the optical character recognition algorithm includes a text detection algorithm and a text recognition algorithm. The training process of the text detection algorithm and the text recognition algorithm is as follows: first, obtain a sample visible light image, a first annotated visible light image, and a second annotated visible light image; wherein the first annotated visible light image is a visible light image of the ladle number text area annotated corresponding to the sample visible light image, and the second annotated visible light image is a visible light image of the ladle number text area annotated corresponding to the sample visible light image, and the ladle number text area annotation may include various fonts, sizes, tilt angles and other different situations under normal circumstances, and the ladle number text annotation may include texts of different fonts and sizes. Then, use Faster R-CNN (Region-based Convolutional Neural Network) to train the text detection algorithm to identify the ladle number text area; use CRNN (Convolutional Recurrent Neural Network) to train the text recognition algorithm to recognize the text in the text area identified by the text detection algorithm; the two algorithms are combined to obtain the final optical character recognition algorithm for identifying the ladle number. During the training process, cross-validation techniques can be used to evaluate the performance of the model and make parameter adjustments to improve the accuracy and robustness of the algorithm.

[0030] Reference Figure 2 , shows a schematic flow chart of another embodiment of the ladle identification method provided by the present invention, the method comprising: S201, collecting thermal infrared images and visible light images of the path of the ladle; wherein the difference between the shooting time of the thermal infrared image and the visible light image is less than a preset time difference.

[0031] S202, determining whether the ladle appears in the thermal infrared image, and if the ladle appears in the thermal infrared image, determining whether the position of the ladle in the thermal infrared image meets a preset position requirement.

[0032] Judging whether the position of the ladle in the thermal infrared image meets the preset position requirement may be judging whether the ladle is located at the center position in the thermal infrared image.

[0033] S203, if the ladle appears in the thermal infrared image and the position of the ladle in the thermal infrared image meets the preset position requirement, the visible light image is identified to obtain information recorded on the ladle in the visible light image.

[0034] If the ladle appears in the thermal infrared image and is centered in the thermal infrared image, the visible light image can be identified to obtain the ladle number in the visible light image.

[0035] S204, performing temperature information recognition on the thermal infrared image to obtain temperature information of the ladle; the temperature information includes a temperature matrix.

[0036] If the ladle appears in the thermal infrared image and is centered in the thermal infrared image, the temperature information in the thermal infrared image can be further identified to obtain a temperature matrix. The temperature matrix is ​​used to describe the temperature conditions of various parts of the ladle.

[0037] S205, determining the state of the ladle according to the temperature matrix and a preset temperature evaluation algorithm.

[0038] S206, when the state of the ladle does not meet the preset state requirements, an alarm message is generated.

[0039] S207, record and store target information of the ladle; the target information includes: at least one of the position of the camera device for taking thermal infrared images and visible light images, the visible light image, the thermal infrared image, the ladle number, the temperature matrix and the state of the ladle.

[0040] After the ladle number, temperature information and status are determined, the ladle number, temperature matrix, ladle status, location of the camera used to take thermal infrared images and visible light images and other information can be stored in the database.

[0041] When the embodiment of the present invention recognizes that a ladle appears in a thermal infrared image and the position of the ladle in the thermal infrared image meets the preset position requirement, the temperature of the ladle is further evaluated to detect whether the ladle is in an abnormal state. If the ladle is in an abnormal state, an alarm is issued in time, thereby avoiding production interruptions or safety accidents. At the same time, storing and recording the target information of the ladle can help managers to trace and hold accountable. Once a problem or accident occurs, the manager can accurately understand the relevant operations and situations, which is helpful for subsequent analysis and processing. Based on the analysis results of the data, managers can make more intelligent decisions and adjustments to improve the scientificity and efficiency of production safety management.

[0042] In one embodiment, step S202 may specifically include: identifying the thermal infrared image through a ladle recognition algorithm, and determining whether the ladle appears in the thermal infrared image, and if the ladle appears in the thermal infrared image, whether the position of the ladle in the thermal infrared image meets the preset position requirement; wherein the ladle recognition algorithm is trained in the following manner: obtaining a sample thermal infrared image taken of the route through which the ladle passes, and an annotated thermal infrared image with the ladle area annotated; and training the ladle recognition algorithm based on the sample thermal infrared image and the annotated thermal infrared image through a convolutional neural network. During the training process, a cross-validation technique may be used to evaluate the performance of the model, and parameter adjustments may be made to improve the accuracy and robustness of the algorithm.

[0043] In one embodiment, the preset temperature assessment algorithm can be trained in advance, and the training process is as follows: First, for the sample thermal infrared images taken along the route of the ladle, and the target thermal infrared images with the temperature matrix as the normal label, the target thermal infrared images should contain normal conditions within various temperature ranges. Then, the temperature assessment algorithm is trained by combining the sample thermal infrared images and the target thermal infrared images through a support vector machine. During the training process, cross-validation techniques can be used to evaluate the performance of the model, and parameter adjustments can be made to improve the accuracy and robustness of the algorithm.

[0044] Reference Figure 3 , showing a framework diagram of the ladle identification method provided by the present invention.

[0045] (1) Installation of camera equipment. Install high-definition binocular cameras at both ends of the rails used to transport ladles in the steel mill. The cameras can capture thermal infrared images as well as visible light images. The cameras can be installed at multiple locations to locate the ladles, quickly capture sample thermal infrared images and sample visible light images, and ensure that the camera's field of view can fully capture the passing locomotive transporting the ladles and clearly capture the image of the ladles. The camera should be installed on a stable bracket to ensure image stability. Configure the camera's network connection to ensure that the server can remotely access and monitor the image in real time.

[0046] (2) Server deployment. The collected sample thermal infrared images and sample visible light images are used to train the text detection algorithm, text recognition algorithm, ladle recognition algorithm, and temperature assessment algorithm, and the trained text detection algorithm, text recognition algorithm, ladle recognition algorithm, and temperature assessment algorithm are deployed on the server. High-performance servers and parallel computing technologies can be used to ensure that the algorithm can process large amounts of data quickly and accurately.

[0047] (3) Ladle detection. The server can read the video stream of the thermal infrared image in real time and identify the thermal infrared image in the video stream. When it is inferred that the ladle appears and moves to the center of the screen, the temperature matrix can be extracted from the thermal infrared image, and the temperature matrix and temperature evaluation algorithm can be used to evaluate the status of the ladle. The status of the ladle can be represented by a numerical value. If the ladle status score is significantly lower than the normal threshold, an alarm is generated and the manager is notified that the ladle status may be abnormal. When it is inferred that the ladle appears and moves to the center of the screen, the visible light image taken at the same time as the thermal infrared image can also be obtained, and the ladle number can be identified using the text detection algorithm and the text recognition algorithm. Finally, the ladle number, camera coordinates, temperature matrix information, thermal infrared image and temperature matrix, and ladle status score can be stored in the database.

[0048] Reference Figure 4, showing a schematic structural diagram of an embodiment of a ladle identification system provided by the present invention, the system 400 comprises: a camera device 401 and a server 402, the camera device and the server are in communication connection; The camera device 401 is used to: take a thermal infrared image and a visible light image of the ladle's route, and send the thermal infrared image and the visible light image to the server; wherein the difference between the shooting time of the thermal infrared image and the visible light image is less than a preset time difference; The server 402 is used to: determine whether the ladle appears in the thermal infrared image; if the ladle appears in the thermal infrared image, identify the visible light image to obtain information recorded on the ladle in the visible light image.

[0049] It should be noted that the implementation principle or implementation process of each module of the above system can refer to the embodiment of the above-mentioned ladle identification method, and will not be described one by one here.

[0050] In one embodiment, the present invention further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the steps of any one of the above-mentioned ladle identification methods are implemented.

[0051] In one embodiment, the present invention further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by the processor, the steps of any one of the above-mentioned ladle identification methods are implemented.

[0052] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0053] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A ladle identification method, characterized in that: include: Collecting a thermal infrared image and a visible light image taken of the path of the ladle; wherein the difference between the shooting time of the thermal infrared image and the visible light image is less than a preset time difference; determining whether the ladle appears in the thermal infrared image; If the steel ladle appears in the thermal infrared image, the visible light image is identified to obtain the information recorded on the steel ladle in the visible light image.

2. The ladle identification method according to claim 1, characterized in that: The determining whether the ladle appears in the thermal infrared image comprises: Determine whether the ladle appears in the thermal infrared image, and if the ladle appears in the thermal infrared image, whether the position of the ladle in the thermal infrared image meets the preset position requirement; If the ladle appears in the thermal infrared image, identifying the visible light image includes: If the ladle appears in the thermal infrared image and the position of the ladle in the thermal infrared image meets the preset position requirement, the visible light image is identified.

3. The ladle identification method according to claim 2, characterized in that: The determining whether the ladle appears in the thermal infrared image, and if the ladle appears in the thermal infrared image, whether the position of the ladle in the thermal infrared image meets a preset position requirement, includes: Identify the thermal infrared image by using a ladle recognition algorithm, and determine whether the ladle appears in the thermal infrared image, and if the ladle appears in the thermal infrared image, whether the position of the ladle in the thermal infrared image meets a preset position requirement; The ladle recognition algorithm is trained in the following way: Acquire a sample thermal infrared image taken for the path of the ladle, and a labeled thermal infrared image with the ladle area labeled; The ladle recognition algorithm is trained based on the sample thermal infrared image and the annotated thermal infrared image through a convolutional neural network.

4. The ladle identification method according to claim 1, characterized in that: The information recorded on the ladle includes the ladle number; and the identifying of the visible light image includes: The visible light image is recognized by an optical character recognition algorithm to obtain the ladle number recorded on the ladle in the visible light image.

5. The ladle identification method according to claim 2, characterized in that: The method further comprises: If the ladle appears in the thermal infrared image and the position of the ladle in the thermal infrared image meets the preset position requirement, temperature information recognition is performed on the thermal infrared image to obtain the temperature information of the ladle.

6. The ladle identification method according to claim 5, characterized in that: The temperature information includes a temperature matrix; the method further includes: Determining the state of the ladle according to the temperature matrix and a preset temperature evaluation algorithm; When the state of the ladle does not meet the preset state requirement, an alarm message is generated.

7. The ladle identification method according to claim 6, characterized in that: The information recorded on the ladle includes the ladle number; the method further comprises: If the ladle appears in the thermal infrared image and the position of the ladle in the thermal infrared image meets the preset position requirement, the target information of the ladle is recorded and stored; the target information includes: the position of the camera equipment that takes the thermal infrared image and the visible light image, the visible light image, the thermal infrared image, the ladle number, the temperature matrix and at least one of the state of the ladle.

8. A ladle identification system, characterized in that: include: A camera device and a server, wherein the camera device and the server are communicatively connected; The camera device is used to: take a thermal infrared image and a visible light image of the ladle's route, and send the thermal infrared image and the visible light image to the server; wherein the difference between the shooting time of the thermal infrared image and the visible light image is less than a preset time difference; The server is used to: determine whether the ladle appears in the thermal infrared image; If the steel ladle appears in the thermal infrared image, the visible light image is identified to obtain the information recorded on the steel ladle in the visible light image.

9. An electronic device, characterized in that: comprising a memory and a processor, wherein: The memory is used to store programs; The processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps of the ladle identification method described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer-readable program, wherein when the program or instruction is executed by a processor, the steps of the ladle identification method described in any one of claims 1 to 7 can be implemented.

Images