Method and apparatus for identifying material information during crane operation

Abstract

The present disclosure provides a method and an apparatus for identifying material information during crane operation, which: obtain image data for a material placed therebelow according to the moving position of a crane during operation of the crane; recognize material information displayed on the surface of the material in the obtained image data; and determine whether the recognized material information matches target material information.

Description

Method and device for checking material information during crane operation

[0001] The present disclosure relates to a technology capable of confirming information on a target material during crane operation.

[0002] Character recognition technology is a fundamental technology widely used in our daily lives, and has been in development since the 1920s. In particular, with the advancement of artificial neural network technology since the 2000s, its use has become increasingly widespread. The use of artificial neural networks has been a groundbreaking advancement in character recognition technology, dramatically improving recognition accuracy and expanding its application areas.

[0003] In line with this trend, efforts are being made in the manufacturing process to move beyond the stage of manually reading material information and monitoring work progress by relying on the human eye, and to utilize character recognition technology to automatically read material information and grasp the overall work progress to optimize production.

[0004] In general, in order to monitor the information of plate materials operated by cranes in the manufacturing process line, manual work was often relied on in the past. However, in order to operate the operation stably and efficiently, the application of technology that automatically recognizes information about materials from acquired images based on artificial neural networks is gradually expanding.

[0005] Especially in the manufacturing industry, manual recognition of information printed on products can be difficult due to peeling, tearing, scratching, contamination, or other factors. Furthermore, due to the nature of the production process, character recognition must be performed while the material is in motion, which can lead to significant variations in the characteristics of the characters being recognized. Furthermore, when working in an environment subject to external light sources, character recognition can be difficult due to variations in light intensity and color tone between day and night, clear and cloudy conditions, and backlighting caused by the angle of sunlight.

[0006] Therefore, a specific design is required for a method that can further increase the recognition rate and prepare for cases of misrecognition.

[0007] The present embodiments aim to provide a method and device for checking material information during crane operation, which enables automation and optimization of production logistics by monitoring the status of materials worked on by a crane in real time by improving the recognition rate of material information being worked on in a crane process.

[0008] In order to solve the above-mentioned problem, one embodiment of the present disclosure is provided for a device for checking material information during crane work, the device including an image acquisition unit installed on a crane and acquiring image data for a material placed below, and a control unit controlling the image acquisition unit to acquire image data according to the movement position of the crane, recognizing material information displayed on the surface of the material within the image data, and determining whether the recognized material information matches pre-stored work target material information.

[0009] In addition, another embodiment may provide a method for confirming material information during crane operation, including a step of controlling an image acquisition unit installed in a crane to obtain image data on a material placed below according to a movement position of the crane, a step of recognizing material information displayed on the surface of the material within the image data, and a step of determining whether the recognized material information matches the information on the material to be worked on.

[0010] According to the present embodiment, a method and device for confirming material information during crane operation can be provided, which can more efficiently obtain information on a material as a work target by comparing material information recognized from image data on a material as a work target of a crane with pre-stored work target material information to confirm material information.

[0011] In addition, by placing cameras on the left and right of the crane to obtain images of the material immediately before and after work depending on the moving position of the crane, it is possible to obtain information on the material before hoisting as well as information on the material placed underneath after hoisting.

[0012] In addition, by installing the camera to acquire images from a vertical downward direction, the influence of backlight can be minimized, allowing for stable acquisition of information on the material.

[0013] In addition, when multiple materials are captured, the material is identified using the information of the material at the top of the system, and by comparing whether it matches using the information of the work target material stored in advance in the system, the recognition rate of the material information can be increased.

[0014] In addition, after lifting, the weight of the material lifted from the crane is measured with a load cell, and the expected candidate material is determined based on the measured value, which is then compared with the material information below, thereby enabling dual material information recognition.

[0015] In addition, it is possible to optimize and improve work efficiency by preventing misrecognition of material information, ensuring that the crane operation status matches the system information, automating inspection of crane operation results, and visualizing logistics.

[0016] FIG. 1 is a drawing for explaining the configuration of a material information confirmation device for confirming material information during crane operation according to one embodiment.

[0017] FIG. 2 is a drawing for explaining an operation of acquiring image data according to a moving position of a crane according to one embodiment.

[0018] FIG. 3 is a drawing for explaining an operation of checking material information before lifting work of a crane according to one embodiment.

[0019] FIG. 4 is a diagram for explaining an operation of recognizing material information from image data according to one embodiment.

[0020] FIG. 5 is a diagram for explaining an operation according to a result of recognizing material information according to one embodiment.

[0021] FIG. 6 is a diagram for explaining an operation when multiple materials are included in image data according to one embodiment.

[0022] FIG. 7 is a drawing for explaining an operation of checking material information during a lifting operation of a crane according to one embodiment.

[0023] FIG. 8 is a drawing for explaining an operation of checking material information after a lifting operation of a crane according to one embodiment.

[0024] FIG. 9 is a drawing illustrating a procedure for checking material information during crane operation according to one embodiment.

[0025] FIG. 10 is a drawing for explaining a procedure for checking material information before lifting work of a crane according to one embodiment.

[0026] FIG. 11 is a drawing for explaining a procedure for checking material information during a lifting operation of a crane according to one embodiment.

[0027] FIG. 12 is a drawing for explaining a procedure for checking material information after a lifting operation of a crane according to one embodiment.

[0028] Hereinafter, some embodiments of the present disclosure will be described in detail with reference to exemplary drawings. When adding reference numerals to components in each drawing, identical components may have the same numerals as much as possible even if they are shown in different drawings. In addition, when describing the present embodiments, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the technical idea of ​​the present invention, the detailed description may be omitted. When "includes," "has," "consists of," etc. are used in this specification, other parts may be added unless "only" is used. When a component is expressed in the singular, it may include a case in which the plural is included unless specifically stated otherwise.

[0029] Additionally, terms such as first, second, A, B, (a), (b), etc. may be used to describe components of the present disclosure. These terms are only intended to distinguish the components from other components, and the nature, order, sequence, or number of the components are not limited by the terms.

[0030] In a description of the positional relationship of components, when it is described that two or more components are "connected," "combined," or "connected," it should be understood that the two or more components may be directly "connected," "combined," or "connected," but that the two or more components may also be further "interposed" with another component to be "connected," "combined," or "connected." Here, the other component may be included in one or more of the two or more components that are "connected," "combined," or "connected" to each other.

[0031] In the description of the temporal flow relationship related to components, operation methods, or manufacturing methods, for example, when the temporal or flow relationship is described as “after”, “following”, “next to”, “before”, etc., it may also include cases where it is not continuous, unless “immediately” or “directly” is used.

[0032] Meanwhile, when numerical values ​​or corresponding information (e.g., levels, etc.) for components are mentioned, even without separate explicit description, the numerical values ​​or corresponding information may be interpreted as including an error range that may occur due to various factors (e.g., process factors, internal or external impact, noise, etc.).

[0033] In this regard, in the present disclosure, a crane refers to a machine that lifts heavy objects and moves them up and down or horizontally, and is not limited to a specific crane as long as the technical idea of ​​the present disclosure is applicable. In addition, the expression "the crane moves" in the present disclosure may be used to mean that the hoist of the crane, etc., moves for hoisting work. In addition, in the present disclosure, a material refers to an object to be moved by a crane, and although the description assumes a plate-shaped material, it is not limited to a specific shape of material as long as the technical idea of ​​the present disclosure is applicable. In addition, in the present disclosure, material information refers to character information such as identification information displayed on the surface of the material to distinguish the material. In addition, in the present disclosure, a vertical downward or vertical direction indicates a shooting direction of an image acquisition unit, and includes not only an exact vertical or vertical direction, but also a state in which the image is tilted within a predetermined micro-angle range.

[0034] Hereinafter, a device for checking material information during crane operation according to embodiments of the present disclosure will be described in detail with reference to related drawings.

[0035] FIG. 1 is a drawing for explaining the configuration of a device for checking material information during crane operation according to one embodiment.

[0036] Referring to FIG. 1, a material information confirmation device (100) may include an image acquisition unit (110) installed on a crane and acquiring image data on a material placed below, and a control unit (130) that controls the image acquisition unit to acquire image data according to the movement position of the crane, recognizes material information displayed on the surface of the material within the image data, and determines whether the recognized material information matches pre-stored work target material information. The configuration of the material information confirmation device (100) illustrated in FIG. 1 is an example and is not limited thereto, and other components may be additionally included or some components may be omitted as needed.

[0037] The image acquisition unit (110) can acquire image data regarding the material being worked on during crane operations. For example, the image acquisition unit (110) may be implemented as an image capturing device, such as a camera, capable of acquiring image data. In this regard, the device is not limited to a specific device, as long as it can acquire image data capable of recognizing material information.

[0038] The image acquisition unit (110) may be installed at a location where it can acquire image data on materials placed below the crane during hoisting operations. As an example, the image acquisition unit (110) may be installed vertically downwards on both sides of the crane hoist. However, this is merely an example, and the image acquisition unit (110) may be installed at any location on the crane as long as it can acquire image data on materials placed below the crane during hoisting operations.

[0039] When a crane moves toward a material for lifting work, the image acquisition unit (110) can acquire image data at a location where the material is placed vertically downwards as the crane moves. Accordingly, an image of the material placed downwards can be included in a region of the acquired image data.

[0040] The image acquisition unit (110) can transmit the acquired image data to the control unit (130). The transmission of the image data is based on a wired or wireless communication method and is not limited to a specific method.

[0041] The control unit (130) may be implemented as a processor, controller, microcontroller, microprocessor, other processing device, or computer.

[0042] The control unit (130) can monitor the movement position of the crane and control the image acquisition unit to acquire image data. That is, when the crane moves to a predetermined position for hoisting work, the control unit can control to acquire image data. In this case, the predetermined position may be a position where the material to be hoisted is placed vertically below the image acquisition unit. In other words, the predetermined position may be a position where the image of the material can be displayed in the acquired image data.

[0043] For example, the control unit (130) can analyze the image data acquired from the image acquisition unit to confirm that the crane has moved to the predetermined location described above. To this end, the control unit (130) can analyze whether a material or a pre-installed mark is recognized within the image data. Alternatively, according to another example, the predetermined location can be set as the movement distance of the crane. In this case, the control unit (130) can determine that the crane has moved to the predetermined location when it has moved by the set movement distance. For this purpose, a separate sensor unit can be provided that measures the movement distance of the crane and transmits the data to the control unit.

[0044] The control unit (130) can detect a text area on the surface of a material within the received image data, where material information is displayed. The control unit (130) can recognize material information about the material from the detected text area. As described above, the material information can include text information displayed on the surface of the material, such as identification information about the material.

[0045] For example, an AI algorithm capable of performing optical character recognition (OCR) can be applied to detect character regions within image data and recognize material information. In this case, various AI algorithms, such as machine learning-based algorithms, deep learning-based algorithms, rectification networks (RNs), convolutional neural networks (CNNs), and recurrent neural networks (RNNs), can be applied, as long as the AI ​​algorithm can recognize material information from image data.

[0046] When material information is recognized from the acquired image data, the control unit (130) can compare it with pre-stored target material information to determine whether it matches. To this end, a separate database may be provided that stores target material information according to the crane's work schedule. Furthermore, the control unit (130) can update the database when information regarding changes to the work schedule or target material is received from an external source.

[0047] The control unit (130) can compare the entire string of the recognized material information with the entire string of the work target material information to determine whether they match. For example, if the entire string of the recognized material information and the entire string of the work target material information do not exactly match, the control unit (130) can determine whether they match by comparing the degree of matching with a predetermined reference value. That is, if the degree of matching between the entire string of the recognized material information and the entire string of the work target material information is equal to or greater than the first reference value, the control unit (130) can determine that the recognized material information matches the work target material information.

[0048] Alternatively, according to another example, if the degree of matching between the entire string of recognized material information and the entire string of the work target material information is greater than or equal to the second reference value, the control unit (130) may correct any misrecognized characters in the string of recognized material information based on the work target material information. The control unit (130) may then determine again whether the characters match the work target material information based on the corrected characters.

[0049] Alternatively, according to another example, if the degree of matching between the entire string of recognized material information and the entire string of the work target material information is less than the third reference value, the control unit (130) may report that the material information does not match the work target material information and output a warning alarm. Each reference value above is an example and is not limited thereto, and may be set differently as needed.

[0050] Accordingly, by comparing the material information recognized from the image data of the material being worked on by the crane with the material information stored in advance, information on the material being worked on can be obtained more efficiently.

[0051] Below, each embodiment related to a device for checking material information during crane operation will be described in more detail with reference to related drawings.

[0052] FIG. 2 is a diagram illustrating an operation for acquiring image data according to a moving position of a crane according to one embodiment. FIG. 3 is a diagram illustrating an operation for checking material information before hoisting work of a crane according to one embodiment. FIG. 4 is a diagram illustrating an operation for recognizing material information from image data according to one embodiment. FIG. 5 is a diagram illustrating an operation according to a result of recognizing material information according to one embodiment.

[0053] Referring to FIG. 2, image acquisition units (110a, 110b) may be installed at at least two locations of the crane (10) to acquire image data in a vertical direction. As an example, as illustrated in FIG. 2, image acquisition units (110a, 110b) may be installed on both sides of the crane (10) along the direction in which the crane (10) moves for work. However, this is merely an example, and the number and installation locations of the image acquisition units may be set differently as needed.

[0054] If, unlike in Fig. 2, the shooting direction of the image acquisition unit is installed so that it faces vertically downward of the gripping unit (11), image data is acquired from the side of the material. In this case, strong specular reflection light is generated and incident by internal and external light sources such as crane lighting, ceiling lighting in a workplace such as a warehouse, and natural lighting such as sunlight. In this case, the amount of light may be saturated, making it difficult to distinguish the target object in the image data. In addition, if the product number of the material placed under the rolled material must also be checked in order to confirm the number of materials, a phenomenon may occur in which the material underneath is obscured by the influence of the rolled material.

[0055] Accordingly, as illustrated in FIG. 2, image acquisition units (110a, 110b) may be installed on the left and right sides of the crane (10) to acquire images in a vertical direction. In this case, image data may be acquired in the vertical direction from either of the image acquisition units (110a, 110b) arranged on either side at the moment the crane moves, immediately before and after the lifting operation, rather than during the lifting operation. In this way, when image data is acquired vertically downward, the entire surface of the material can be confirmed while eliminating interference from surrounding objects. In addition, in such an arrangement structure, when acquiring image data, the crane or the lifted material is positioned in the path of the direction of regular reflection toward the image acquisition unit. Accordingly, the regular reflection light is effectively blocked, eliminating the influence of backlight, and thus more accurate material information can be recognized.

[0056] The crane (10) can move in the direction of the material (20) placed below in order to lift the material (20). Referring to FIG. 3, when the crane (10) moves to a location where the material (20) is placed for lifting, the control unit (130) can control the crane (10) to acquire image data at a location where the material is placed vertically below the image acquisition unit (110a).

[0057] Referring to FIG. 4, the control unit (130) can detect the position of the product number and recognize characters in the detected area (30) in order to recognize material information such as a product number from the surface image of the material (20) in the acquired image data. That is, the control unit (130) can apply an artificial intelligence algorithm to the image data to detect a character area (30) on the surface of the material (20) where material information is displayed, and recognize the material information displayed in the detected character area.

[0058] In addition, the control unit (130) can determine whether the material information recognized in the acquired image data matches the material information to be worked on. In relation to character recognition, the entire string of the material information can be recognized, and the control unit (130) can compare the entire string of the recognized material information to see how much it matches the string of the material information to be worked on, which is the product number information on the system. In this case, the control unit (130) can recognize it as a true value if the degree of matching exceeds a predetermined reference value. Accordingly, the recognition of the material information becomes simple as a case of binary classification of matching or mismatching with the material information to be worked on. At this time, the algorithm for determining the reference value can be determined in various ways. For example, even if one or two characters are misrecognized, the product number of the worked material can be estimated by comparing it with the product number on the system using the remaining characters.

[0059] Referring to Fig. 5, examples of material information recognition results for the detected area (30) are illustrated. In this case, it is assumed that the product number, which is the pre-stored work target material information, is PC146208-03, which is a total of 11 digits.

[0060] For example, the control unit (130) may determine that the recognized material information matches the work target material information if the degree of matching between the entire string of the recognized material information and the entire string of the work target material information is greater than or equal to a first reference value. For example, among the recognition results of FIG. 5, the first example corresponds to a case where character recognition was accurately performed as PC146208-03, and therefore the control unit (130) may determine the recognition result as material information as is.

[0061] Also, in the second example, it may be recognized as PO1462D8-03, and it may be determined that two characters, C and 0, were incorrectly recognized. In this case, since 9 out of 11 characters in the entire string match, the matching degree is approximately 81%. If the first reference value is set to 80%, the control unit (130) may determine that the recognized material information corresponds to the work target material information.

[0062] For example, if the degree of matching between the entire string of recognized material information and the entire string of the work target material information is equal to or greater than a second reference value, the control unit (130) may correct misrecognized characters in the string of recognized material information based on the work target material information. For example, if the second reference value is set to 80%, the control unit (130) may correct mismatched characters, such as C and 0, in the recognized material information based on the work target material information. In this case, for example, the control unit (130) may determine the degree of similarity between mismatched characters by applying a predetermined artificial intelligence algorithm. If the degree of similarity between the mismatched characters is determined to be lower than a predetermined reference value, the control unit (130) may stop correcting the misrecognized characters.

[0063] For example, the control unit (130) may output a warning alarm if the degree of agreement between the entire string of recognized material information and the entire string of work target material information is less than the third reference value. For example, in the case of the third example of FIG. 5, it may be recognized as OCL46SDE-PB, and it may be determined that a total of 7 characters do not match each other. In this way, if a recognition result that is completely different from the work target material information on the system appears and it is impossible to determine whether they are the same, the control unit (130) may sound an alarm so that the manager can check the work result.

[0064] In this way, the pre-loaded material is compared with the target material information in the system, and if there's a match, it's considered material information. By comparing the recognized material information with the target material information in the system on a one-to-one basis, the problem can be simplified and the accuracy of material recognition can be improved. This allows the workpiece information to be easily compared with the system information even when the material surface is contaminated, rusted, or has incomplete markings.

[0065] For example, for more accurate judgment, image data may be acquired in multiple pieces for a material (20). That is, multiple image data may be acquired for the same material, and material information may be recognized from each image data.

[0066] In this case, the control unit (130) can determine whether the material information recognized in each of the plurality of image data matches the information on the work-target material, and if the material information recognized in at least one image among the plurality of image data matches the information on the work-target material, the material (20) can be determined as the work-target material. If there is no result matching the information on the work-target material among the plurality of image data, the control unit (130) can determine that the recognition has failed and output a warning alarm.

[0067] FIG. 6 is a diagram for explaining an operation when multiple materials are included in image data according to one embodiment.

[0068] As illustrated in Fig. 6, when plate-like materials of different sizes are stacked or when plate-like materials are stacked in a disordered state, text areas (30, 31) displaying material information for each of the plurality of materials (20, 21) may be included in the image data. When material information for each of the plurality of materials is recognized in the image data, the control unit (130) may determine the material located at the topmost position among the plurality of materials based on the information on the work target material.

[0069] For example, let's assume that the work target material information pre-stored in the system is PC146208-03. Referring to Fig. 6, the two recognized strings are respectively represented as PC146208-03 in the character area (30) and PC146209-01 in the character area (31). Since the product number of the material that is the target of the next winding operation is PC146208-03, the control unit (130) can determine the material corresponding to the material information recognized in the character area (30) as the material located at the top. In addition, as described below, the control unit (130) can recognize the material information (31) for the material (21) remaining below after winding, and determine that the decision was appropriate.

[0070] FIG. 7 is a drawing for explaining an operation of checking material information during a hoisting operation of a crane according to one embodiment. FIG. 8 is a drawing for explaining an operation of checking material information after a hoisting operation of a crane according to one embodiment.

[0071] Referring to FIG. 7, after the crane (10) hoists the material (20), the control unit (130) can compare the weight of the load cell of the crane with the weight information of the material to be worked on, thereby determining at least one expected material candidate for the material placed below. At this time, the control unit (130) can compare the weight information of the load cell indicating the weight of the hoisted material with the information on the materials to be worked on in the system. Through this, the control unit (130) can roughly estimate the number of hoists and select 1 to 2 expected material candidates remaining below.

[0072] Referring to FIG. 8, the control unit (130) can control to acquire image data at a position where the material (21) is placed vertically below the image acquisition unit (110a) when the crane (10) moves the hoisted material (20). In FIG. 8, it is illustrated on the premise that the crane (10) returns to the right after hoisting, but it is not limited thereto. When the crane (10) continues to move to the left after hoisting, the image data of the material (21) placed below can be acquired by the image acquisition unit (110b) on the right.

[0073] In this case, the control unit (130) can determine whether the material information recognized in the acquired image data matches the material information of at least one expected material candidate. In this way, in the case of a material after winding, the expected material candidates are compressed to 1 or 2 based on the load cell measurement values, and if the material information among them matches, the material information is updated. If the discrimination power of the matching rate falls below a certain level as a result of comparing with 1 or 2 expected candidate materials, the control unit (130) can sound an alarm so that the manager can check the work result.

[0074] This allows for easy estimation of workpiece information even if some errors occur in the material information. Furthermore, by preventing misidentification and ensuring that crane operation status matches system information, it automates crane operation result inspection and visualizes logistics, optimizing and improving operational efficiency.

[0075] After the lifting operation on the material (20) is completed, if the crane (10) performs the lifting operation again on the material (21), the aforementioned operation may be repeated. Accordingly, for the same material, recognition may be performed during the previous lifting operation, and recognition may be performed again during the current lifting operation. The control unit (130) may compare the results of the matching determination in the two cases and output an alarm if they are different. In this way, by performing the matching determination twice on the same material, the false recognition rate may be reduced.

[0076] Accordingly, by comparing the material information recognized from the image data of the material being worked on by the crane with the material information stored in advance, information on the material being worked on can be obtained more efficiently.

[0077] Hereinafter, a method for confirming material information during crane operation, which can perform some or all of the embodiments described with reference to FIGS. 1 through 8, will be described with reference to the drawings. The above description may be omitted to avoid redundant explanation. In this case, the omitted content may be substantially identically applied to the following description, as long as it does not contradict the technical spirit of the invention.

[0078] FIG. 9 is a drawing illustrating a procedure (900) of a method for checking material information during crane operation according to one embodiment.

[0079] Referring to FIG. 9, the material information confirmation device can control an image acquisition unit installed on the crane to acquire image data on a material placed below according to the movement position of the crane (S910).

[0080] An image acquisition unit can acquire image data of a material to be worked on during crane operation. The image acquisition unit can be installed at a location where it can acquire image data of a material placed below the crane during hoisting operation. When the crane moves toward the material for hoisting operation, the image acquisition unit can acquire image data at a location where the material is placed vertically below the crane as the crane moves. Accordingly, an image of the material placed below can be included in a portion of the acquired image data.

[0081] The material information confirmation device can monitor the movement position of the crane and control the image acquisition unit to acquire image data. That is, when the crane moves to a predetermined position for hoisting, the control can be performed to acquire image data. In this case, the predetermined position may be a position where the material to be hoisted is located vertically below the image acquisition unit. In other words, the predetermined position may be a position where an image of the material can be displayed in the acquired image data.

[0082] For example, the material information verification device can analyze image data acquired from the image acquisition unit to verify that the crane has moved to the predetermined location described above. To this end, the material information verification device can analyze whether a material or a pre-installed mark is recognized in the image data. Alternatively, according to another example, the predetermined location can be set as the movement distance of the crane. In this case, the material information verification device can determine that the crane has moved to the predetermined location when the crane has moved by the set movement distance. To this end, a separate sensor unit can be provided that measures the movement distance of the crane and transmits the measured data to the control unit.

[0083] Referring again to FIG. 9, the material information confirmation device can recognize material information displayed on the surface of a material within image data (S920) and determine whether the recognized material information matches the pre-stored work target material information (S930).

[0084] A material information verification device can detect a text area on the surface of a material within the received image data, where material information is displayed. The material information verification device can recognize material information about the material from the detected text area. As described above, the material information may include text information displayed on the surface of the material, such as identification information about the material.

[0085] When the material information verification device recognizes material information from the acquired image data, it can compare it with pre-stored target material information to determine whether it matches. To this end, a separate database can be provided that stores target material information based on the crane's work schedule. Furthermore, the material information verification device can update the database when information about changes to the work schedule or target material is received from an external source.

[0086] The material information verification device can compare the entire string of the recognized material information with the entire string of the work target material information to determine whether there is a match. For example, if the entire string of the recognized material information and the entire string of the work target material information do not exactly match, the material information verification device can determine whether there is a match by comparing the degree of match with a predetermined reference value. That is, if the degree of match between the entire string of the recognized material information and the entire string of the work target material information is equal to or greater than the first reference value, the material information verification device can determine that the recognized material information matches the work target material information.

[0087] Alternatively, according to another example, if the degree of matching between the entire string of recognized material information and the entire string of the work target material information is greater than or equal to the second reference value, the material information verification device may correct any misrecognized characters in the string of recognized material information based on the work target material information. The material information verification device may then determine whether the characters match the work target material information again based on the corrected characters.

[0088] Alternatively, according to another example, if the degree of agreement between the entire string of recognized material information and the entire string of target material information is less than a third reference value, the material information verification device may report that the material information does not match the target material information and output a warning alarm. Each reference value above is an example and is not limited thereto, and may be set differently as needed.

[0089] Accordingly, by comparing the material information recognized from the image data of the material being worked on by the crane with the material information stored in advance, information on the material being worked on can be obtained more efficiently.

[0090] FIG. 10 is a diagram illustrating a procedure (1000) for checking material information before hoisting operation of a crane according to one embodiment. FIG. 11 is a diagram illustrating a procedure (1100) for checking material information during hoisting operation of a crane according to one embodiment. FIG. 12 is a diagram illustrating a procedure (1200) for checking material information after hoisting operation of a crane according to one embodiment.

[0091] Referring to FIG. 10, the material information verification device can compare the camera position with the material position during the crane's movement (S1010). By monitoring the crane's moving position, the device can control the camera to acquire image data. The material information verification device can control the camera to capture multiple images when the position of one of the cameras installed on the crane approaches the material position (S1020).

[0092] The material information verification device can detect a product number object from each of the acquired multiple video images (S1030) and recognize the product number from the detected object (S1040). That is, the material information verification device can apply an artificial intelligence algorithm to the image data to detect a text area on the surface of the material within the image data where material information is displayed, and recognize the material information displayed in the detected text area.

[0093] When multiple product numbers are recognized in a single image, the material information verification device can determine the product number that best matches the expected material number (1) in the system by comparing it to the expected material number (S1050). For example, when plate-shaped materials of different sizes are stacked or when plate-shaped materials are stacked in a disordered state, text areas indicating material information for each of the multiple materials can be included in the image data. The material information verification device can determine the material located at the topmost position among the multiple materials based on the target material information.

[0094] The material information verification device can determine whether the material information recognized from the acquired image data matches the target material information. In terms of character recognition, the entire string of material information can be recognized, and the material information verification device can compare the entire string of recognized material information to the string of target material information, which is the product number information in the system, to determine how closely it matches.

[0095] If the material information verification device does not completely match the expected material number (1) in the system, the device can correct the error and determine the target material (S1060). If the degree of matching between the entire string of the recognized material information and the entire string of the material information to be worked on is greater than or equal to a first reference value, the device can determine that the recognized material information matches the material information to be worked on. If the degree of matching between the entire string of the recognized material information and the entire string of the material information to be worked on is greater than or equal to a second reference value, the device can correct any misrecognized characters in the string of the recognized material information based on the material information to be worked on.

[0096] The material information verification device can determine the target material if there is a case where the expected material number (1) in the system matches one of the multiple images (S1070). The material information verification device determines whether the material information recognized in each of the multiple image data matches the information on the target material for work, and if the material information recognized in at least one of the multiple image data matches the information on the target material for work, the material can be determined as the target material for work.

[0097] The material information verification device may generate a warning alarm if none of the multiple images match the expected material number (1) in the system (S1080). That is, the material information verification device may output a warning alarm if the degree of matching between the entire string of recognized material information and the entire string of target material information is less than a third reference value.

[0098] Referring to Fig. 11, the material information verification device can select one or two candidates for expected materials placed below after the lifting operation by comparing the weight of the load cell during lifting with the weight of materials in the system after the lifting operation (S1110). The material information verification device can compare the weight information of the load cell, which indicates the weight of the lifted material, with the information on the materials to be worked on in the system. Through this, the material information verification device can roughly estimate the number of lifting orders and select one or two candidates for expected materials remaining below.

[0099] Referring to FIG. 12, the material information verification device can compare the camera position with the material position during the crane's movement (S1210). By monitoring the crane's moving position, the device can control the camera to acquire image data. The material information verification device can control the camera to capture multiple images when the position of one of the cameras installed on the crane approaches the material position (S1220).

[0100] The material information verification device can detect a product number object from each of the acquired multiple video images (S1230) and recognize the product number from the detected object (S1240). That is, the material information verification device can apply an artificial intelligence algorithm to the image data to detect a text area on the surface of a material within the image data where material information is displayed, and recognize the material information displayed in the detected text area.

[0101] When multiple product numbers are recognized in a single image, the material information verification device can determine the product number that best matches the expected material candidate (1-2) by comparing it with the expected material candidate (S1250). For example, when plate-shaped materials of different sizes are stacked or when plate-shaped materials are stacked in a disordered state, text areas indicating material information for each of the multiple materials may be included in the image data. The material information verification device can determine the material located at the topmost position among the multiple materials based on the target material information.

[0102] If the material information verification device does not completely match the expected material number (1-2) in the system, the device can correct the error and determine the target material (S1260). If the degree of matching between the entire string of the recognized material information and the entire string of the material information to be worked on is greater than or equal to a first reference value, the device can determine that the recognized material information matches the material information to be worked on. If the degree of matching between the entire string of the recognized material information and the entire string of the material information to be worked on is greater than or equal to a second reference value, the device can correct any misrecognized characters in the string of the recognized material information based on the material information to be worked on.

[0103] The material information verification device can determine that a material is a target material if it matches the expected material numbers (1-2) in the system from multiple images and update the system (S1270). The material information verification device can determine whether the material information recognized in each of the multiple image data matches the information on the target material for work, and can determine that the material is a target material for work if the material information recognized in at least one image among the multiple image data matches the information on the target material for work.

[0104] The material information verification device may generate a warning alarm if none of the multiple images match the expected material numbers (1-2) in the system (S1280). That is, the material information verification device may output a warning alarm if the degree of agreement between the entire string of recognized material information and the entire string of target material information is less than a third reference value.

[0105] Accordingly, by comparing the material information recognized from the image data of the material being worked on by the crane with the material information stored in advance, information on the material being worked on can be obtained more efficiently.

[0106] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will readily appreciate that the present invention can be readily modified into other specific forms without altering the technical spirit or essential characteristics of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single entity may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined manner.

[0107] The scope of the present invention is indicated by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention.

[0108] The above description is merely an illustrative example of the technical idea of ​​the present disclosure, and those skilled in the art to which the present disclosure pertains will appreciate that various modifications and variations can be made without departing from the essential characteristics of the technical idea of ​​the present disclosure. In addition, the present embodiments are not intended to limit the technical idea of ​​the present disclosure but rather to explain it, and therefore the scope of the technical idea of ​​the present disclosure is not limited by these embodiments. The scope of protection of the present disclosure should be interpreted by the claims below, and all technical ideas within a scope equivalent thereto should be interpreted as being included within the scope of the rights of the present disclosure.

[0109]

[0110] CROSS-REFERENCE TO RELATED APPLICATION

[0111] This patent application claims priority under 35 USC § 119(a) to Korean Patent Application No. 10-2023-0179681, filed December 12, 2023, the entire contents of which are incorporated herein by reference. Furthermore, this patent application claims priority in countries other than the United States for the same reasons, the entire contents of which are incorporated herein by reference.

Claims

1. In a device for checking material information during crane operation, An image acquisition unit installed on a crane and acquiring image data on materials placed below; and A material information confirmation device including a control unit that controls the image acquisition unit to acquire the image data according to the movement position of the crane, recognizes material information displayed on the surface of the material within the image data, and determines whether the recognized material information matches the pre-stored work target material information.

2. In paragraph 1, The above image acquisition unit, A material information confirmation device installed at at least two locations of the crane to obtain vertically downward image data.

3. In paragraph 1, The above control unit, A material information confirmation device that applies an artificial intelligence algorithm to the above image data to detect a character area on the surface of a material in which material information is displayed, and recognizes the material information displayed in the detected character area.

4. In paragraph 3, The above control unit, A material information verification device that determines that the recognized material information matches the work target material information when the degree of matching between the entire string of recognized material information and the entire string of work target material information is greater than or equal to a first reference value.

5. In paragraph 3, The above control unit, A material information verification device that corrects misrecognized characters in a string of material information recognized based on the material information to be worked on, when the degree of matching between the entire string of recognized material information and the entire string of material information to be worked on is greater than or equal to a second reference value.

6. In paragraph 3, The above control unit, A material information verification device that outputs a warning alarm when the degree of matching between the entire string of recognized material information and the entire string of work target material information is less than the third reference value.

7. In paragraph 1, The above control unit, A material information confirmation device that determines the material located at the top among a plurality of materials based on the work target material information when material information for each of a plurality of materials is recognized within the above image data.

8. In paragraph 1, The above image data is acquired in multiple pieces for the above material, The above control unit, A material information confirmation device that determines whether material information recognized in each of the plurality of image data matches the work target material information, and determines the material as the work target material if the material information recognized in at least one image among the plurality of image data matches the work target material information.

9. In paragraph 1, The above control unit, A material information confirmation device that controls the image acquisition unit to acquire image data at a location where the material is placed vertically below the image acquisition unit when the crane moves to a location where the material is placed for lifting work, and determines whether the material information recognized in the acquired image data matches the information on the material to be worked on.

10. In paragraph 1, The above control unit, A material information confirmation device that compares the weight of the load cell of the crane and the weight information of the material to be worked on after the crane lifts the material, and determines at least one expected material candidate for the material placed below.

11. In Article 10, The above control unit, A material information confirmation device that controls the crane to acquire image data at a location where the material is placed vertically below the image acquisition unit when the crane moves the material, and determines whether the material information recognized in the acquired image data matches the material information of the at least one expected material candidate.

12. In the method of checking material information during crane operation, A step of controlling an image acquisition unit installed in the crane to acquire image data on a material placed below according to the moving position of the crane; A step of recognizing material information displayed on the surface of a material within the above image data; and A method for verifying material information, comprising: a step of determining whether recognized material information matches pre-stored work target material information.

13. In paragraph 12, The above image acquisition unit, A material information confirmation method installed at at least two locations of the crane to obtain vertically downward image data.

14. In paragraph 12, The step of recognizing the above material information is: A material information confirmation method that applies an artificial intelligence algorithm to the above image data to detect a character area on the surface of a material in which material information is displayed, and recognizes the material information displayed in the detected character area.

15. In paragraph 14, The step of determining whether the above matches is as follows: A material information verification method for determining that the recognized material information matches the work target material information when the degree of matching between the entire string of recognized material information and the entire string of work target material information is greater than or equal to a first criterion value.

16. In paragraph 14, The step of determining whether the above matches is as follows: A material information verification method for correcting misrecognized characters in a string of material information recognized based on the material information to be worked on, when the degree of matching between the entire string of recognized material information and the entire string of material information to be worked on is greater than or equal to a second reference value.

17. In paragraph 14, A material information verification method further comprising a step of outputting a warning alarm when the degree of matching between the entire string of recognized material information and the entire string of work target material information is less than a third reference value.

18. In paragraph 12, The step of determining whether the above matches is as follows: A material information confirmation method for determining the material located at the top among a plurality of materials based on the work target material information when material information for each of a plurality of materials is recognized within the above image data.

19. In paragraph 12, The above image data is acquired in multiple pieces for the above material, The step of determining whether the above matches is as follows: A material information confirmation method for determining whether material information recognized in each of the plurality of image data matches material information to be worked on, and determining the material as the material to be worked on if the material information recognized in at least one image among the plurality of image data matches the material information to be worked on.

20. In paragraph 12, The above controlling step is, When the above crane moves to a location where the material is placed for lifting work, it controls to acquire image data at a location where the material is placed vertically below the image acquisition unit, The step of determining whether the above matches is as follows: A material information verification method for determining whether the material information recognized from acquired image data matches the material information to be worked on.

Images