Vision-based field bridge positioning method, device and system and electronic equipment

Abstract

The embodiment of the invention provides a field bridge positioning method, device and system based on vision and electronic equipment. The method comprises the steps that a digital identifier image of an operation position of a yard bridge is acquired, the digital identifier image comprises at least two digital identifiers, and the digital identifiers are arranged along a lane line corresponding to the operation position; determining a first center coordinate of a digital identifier in the digital identifier image and a number in the digital identifier; determining a target deviation distance of the first center coordinate relative to a second center coordinate of the digital identifier image; and if the number meets the preset continuity requirement, positioning the field bridge according to the target deviation distance. According to the method, position information of the field bridge is supplemented through visual information, so that field bridge positioning is more accurate.

Description

Technical Field

[0001] This application relates to the field of automation technology, and in particular, to a vision-based positioning method, device, system, and electronic device for a gantry crane. Background Art

[0002] When goods need to be transported, the gantry crane is controlled by the control system instructions to move to the designated loading and unloading area to grab the goods to be transported, and then transfer the goods to the destination. During this process, the gantry crane generally relies on an automated control system for precise positioning and navigation to ensure that it moves along the established track and avoids collisions and misoperations during the transportation process.

[0003] During the operation of the gantry crane, the position information of the gantry crane is usually determined by the Global Positioning System (GPS) installed on the gantry crane.

[0004] However, the GPS signal may be interfered by the surrounding environment, resulting in insufficient positioning accuracy. Summary of the Invention

[0005] Embodiments of this application provide a vision-based positioning method, device, system, and electronic device for a gantry crane to achieve the effect of improving the positioning accuracy of the gantry crane.

[0006] In a first aspect, embodiments of this application provide a vision-based positioning method for a gantry crane, including:

[0007] Obtain a digital identifier image of the working position of the gantry crane, where the digital identifier image contains at least two digital identifiers, and the digital identifiers are arranged along the lane line corresponding to the working position;

[0008] Determine the first center coordinates of the digital identifiers in the digital identifier image and the numbers in the digital identifiers;

[0009] Determine the target deviation distance of the first center coordinates relative to the second center coordinates of the digital identifier image;

[0010] If the numbers meet the preset continuity requirements, position the gantry crane according to the target deviation distance.

[0011] In a possible implementation manner, positioning the gantry crane according to the target deviation distance includes:

[0012] Determine the scaling ratio of the digital identifier image according to the built-in parameters of the image acquisition device, where the image acquisition device is used to acquire the digital identifier image, and the image acquisition device is arranged on the bracket of the gantry crane;

[0013] Based on the scaling ratio, perform scaling processing on the target deviation distance to obtain the actual deviation distance corresponding to the target deviation distance;

[0014] Determine the position of the gantry crane according to the actual deviation distance.

[0015] In a possible implementation, the setting spacing between adjacent digital identifiers is less than a target value, and the target value is the sum of the horizontal visual width of the image acquisition device and the horizontal length of the digital identifier.

[0016] In a possible implementation, the digital identifiers are arranged vertically along the lane line, the shape of the digital identifiers is a symmetric figure, and the color contrast between the digital area and the non-digital area in the digital identifiers is greater than a preset contrast threshold.

[0017] In a possible implementation, before positioning the gantry crane according to the number and the target deviation distance, the method further includes:

[0018] Determine the difference value between the target deviation distance and the standard deviation distance according to the target deviation distance, where the standard deviation distance is the deviation distance between the first center coordinate and the second center coordinate when the gantry crane is in the standard operation position;

[0019] Determine that the difference value is less than or equal to a preset difference value threshold.

[0020] In a possible implementation, the method further includes:

[0021] If the difference value is greater than the difference value threshold, it is determined that the gantry crane positioning fails, and a positioning failure message is generated, where the positioning failure message includes whether the gantry crane deviates from the lane line and / or whether the angle of the image acquisition device on the gantry crane changes.

[0022] In a possible implementation, determining the first center coordinate of the digital identifier in the digital identifier image and the number in the digital identifier includes:

[0023] Based on a preset target detection model, identify the digital identifier image to obtain the digital identifier area in the digital identifier image and the category information corresponding to the digital identifier area;

[0024] Determine the center point coordinate of the recognition frame corresponding to the digital identifier area according to the digital identifier area;

[0025] Determine the number corresponding to the category information according to the category information corresponding to the digital identifier area;

[0026] Determine the first center coordinate of the digital identifier and the number in the digital identifier according to the center point coordinate of the recognition frame and the number corresponding to the category information.

[0027] In a second aspect, an embodiment of the present application provides a vision-based gantry crane positioning device, including:

[0028] An acquisition module, configured to acquire a digital identifier image of the working position of a gantry crane, where the digital identifier image includes at least two digital identifiers, and the digital identifiers are arranged along the lane line corresponding to the working position;

[0029] A first determination module, configured to determine the first central coordinates of the digital identifiers in the digital identifier image and the digits in the digital identifiers;

[0030] A second determination module, configured to determine the target deviation distance of the first central coordinates relative to the second central coordinates of the digital identifier image;

[0031] A positioning module, configured to, if the digits meet the preset continuity requirement, position the gantry crane according to the target deviation distance.

[0032] In a possible implementation manner, the positioning module is specifically configured to:

[0033] Determine whether at least two digital identifiers meet the preset setting requirements according to the digits;

[0034] If at least two digital identifiers meet the preset setting requirements, position the gantry crane according to the target deviation distance.

[0035] In a possible implementation manner, the positioning module is further configured to:

[0036] Determine the scaling ratio of the digital identifier image according to the built-in parameters of the image acquisition device, where the image acquisition device is used to acquire the digital identifier image, and the image acquisition device is arranged on the bracket of the gantry crane;

[0037] Based on the scaling ratio, perform a scaling process on the target deviation distance to obtain the actual deviation distance corresponding to the target deviation distance;

[0038] Determine the position of the gantry crane according to the actual deviation distance.

[0039] In a possible implementation manner, the second determination module is further configured to:

[0040] Determine the difference value between the target deviation distance and the standard deviation distance according to the target deviation distance, where the standard deviation distance is the deviation distance between the first central coordinates and the second central coordinates when the gantry crane is at the standard working position;

[0041] Determine that the difference value is less than or equal to the preset difference value threshold.

[0042] In a possible implementation manner, the second determination module is further configured to:

[0043] If the difference value is greater than the difference value threshold, it is determined that the positioning of the yard crane fails, and a positioning failure message is generated. The positioning failure message includes whether the yard crane deviates from the lane line and / or whether the angle of the image acquisition device on the yard crane changes.

[0044] In a possible implementation manner, the first determination module is specifically configured to:

[0045] Based on a preset target detection model, identify the digital identifier image to obtain the digital identifier region in the digital identifier image and the category information corresponding to the digital identifier region;

[0046] According to the digital identifier region, determine the center point coordinates of the recognition frame corresponding to the digital identifier region;

[0047] According to the category information corresponding to the digital identifier region, determine the number corresponding to the category information;

[0048] According to the center point coordinates of the recognition frame and the number corresponding to the category information, determine the first center coordinates of the digital identifier and the number in the digital identifier.

[0049] In a third aspect, an embodiment of the present application provides a vision-based yard crane positioning system, including:

[0050] An image acquisition device for acquiring a digital identifier image of the working position of the yard crane;

[0051] An electronic device for implementing the above first aspect and / or various possible implementation manners of the first aspect.

[0052] In a fourth aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;

[0053] The memory stores computer execution instructions;

[0054] The processor executes the computer execution instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementation manners of the first aspect.

[0055] The vision-based yard crane positioning method, device, system and electronic device provided by the embodiments of the present application obtain a digital identifier image including at least two digital identifiers, determine the first center coordinates of the digital identifier in the digital identifier image, and at the same time identify the number in the digital identifier, and then calculate the target deviation distance between the first center coordinates and the second center coordinates. If the number meets the preset continuity requirement, the position of the yard crane can be determined according to the target deviation distance, supplement the position information of the yard crane through visual information, and verify the accuracy of the identification through the continuity of the number, which can ensure more accurate positioning of the yard crane and reduce the working position error of the yard crane. Description of the Drawings

[0056] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0057] Figure 1 Flow schematic of the vision-based yard crane positioning method provided by an embodiment of the present application Figure 1 ;

[0058] Figure 2 Schematic diagram of the digital identifier image provided by an embodiment of the present application;

[0059] Figure 3 Top view of the yard crane operation scenario provided by an embodiment of the present application;

[0060] Figure 4 Front view of the yard crane operation scenario provided by an embodiment of the present application;

[0061] Figure 5 Flow schematic of the vision-based yard crane positioning method provided by an embodiment of the present application Figure 2 ;

[0062] Figure 6 Schematic structural diagram of the vision-based yard crane positioning device provided by an embodiment of the present application;

[0063] Figure 7 Schematic structural diagram of the vision-based yard crane positioning system provided by an embodiment of the present application;

[0064] Figure 8 Schematic structural diagram of the electronic device provided by an embodiment of the present application.

[0065] Through the above accompanying drawings, specific embodiments of the present application have been shown, and there will be more detailed descriptions hereinafter. These accompanying drawings and written descriptions are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. Detailed implementation manners

[0066] Here, the exemplary embodiments will be described in detail, and the examples are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present application. On the contrary, they are only examples of the devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0067] First, the nouns involved in the present application are explained:

[0068] Quayside crane: A large-scale lifting equipment used for loading and unloading containers or other goods, usually used in ports, logistics centers, yards and other locations. The quayside crane is usually erected above the cargo yard, and uses the bridge structure and spreader to lift the goods through steel cables or other mechanical devices and move them to the designated position.

[0069] In the prior art, there are various methods for positioning the quayside crane. For example, the geographical location of the quayside crane is determined through GPS signals, or the magnet induction signals output by the detection and counting device are used, etc. However, these positioning methods are usually based on signal monitoring. When the external environment affects the signals, it may lead to unstable signals, thereby affecting the positioning accuracy.

[0070] Based on this, the vision-based quayside crane positioning method provided in this application, because the signals are vulnerable to environmental influences and cannot be directly visible. Therefore, if digital identifiers that are easy to identify can be set for the working environment of the quayside crane, visual information can be used for basic positioning in a complex environment. Specifically, if the digital identifiers are set according to the working position of the quayside crane, it can adapt to different site layouts and environmental changes. At the same time, since the numbers in the digital identifiers are usually unique, specific positions can be corresponding to different numbers, so as to quickly identify and distinguish different working positions. In addition, the digital identifier image taken for the working position has a certain proportional relationship with the actual working environment. Therefore, if the digital identifier and its position in the image can be identified, the offset of the quayside crane relative to a specific position can be accurately determined, thereby accurately positioning the quayside crane. Thus, reliable positioning information can be provided in different environments through this method, further improving the automation level of the quayside crane.

[0071] The technical solution of this application and how the technical solution of this application solves the above technical problems will be described in detail below with specific embodiments. These several specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0072] Figure 1 Flow schematic of the vision-based quayside crane positioning method provided by the embodiments of this application Figure 1 . As Figure 1 shown, the method may include the following steps:

[0073] S101. Obtain a digital identifier image of the working position of the quayside crane. The digital identifier image contains at least two digital identifiers, and the digital identifiers are arranged along the lane line corresponding to the working position.

[0074] Among them, the working position of the gantry crane is usually near the bay. A bay is a location in a port, cargo yard, or container terminal used to identify the storage location of containers or goods. The digital identifier image can refer to an image obtained through a camera or other visual device, which shows the bay at the working position of the gantry crane. The image contains at least two digital identifiers, which are set along the lane line corresponding to the working position and are used to assist in positioning. The lane line can refer to the marking line on the road at the working position, which is used to indicate the path of vehicle travel.

[0075] In this embodiment, the digital identifier can be a combined identifier of an outer contour graph and a number. For example, the number can be an Arabic numeral such as 1, 2, 3, etc., which is used to identify a specific working position or lane. When these digital identifiers are set, they have a specific spatial layout, and the adjacent digital identifiers usually have the same spacing.

[0076] Exemplarily, the digital identifier can be hung on the fence beside the bay along the lane line, so as to reduce wear and lower the maintenance cost. If there is no fence beside the bay, the digital identifier can be made into a standing sign and placed vertically along the lane line to prevent the influence of extreme weather such as rain, snow, etc., resulting in difficult recognition.

[0077] S102. Determine the first center coordinates of the digital identifiers in the digital identifier image and the numbers in the digital identifiers.

[0078] Among them, the first center coordinates can refer to the coordinates of the center point of the digital identifier. Since the digital identifier image contains at least two digital identifiers, correspondingly, at least two first center coordinates also need to be determined.

[0079] In this embodiment, the deep learning method can be used, and the convolutional neural network (CNN) is used for the detection and recognition of digital identifiers.

[0080] In a possible implementation manner, the specific implementation manner of step S102 can be:

[0081] Based on a preset object detection model, identify the digital identifier image to obtain the digital identifier region in the digital identifier image and the category information corresponding to the digital identifier region; then, according to the digital identifier region, determine the center point coordinates of the recognition frame corresponding to the digital identifier region; furthermore, according to the category information corresponding to the digital identifier region, determine the number corresponding to the category information; thus, according to the center point coordinates of the recognition frame and the number corresponding to the category information, determine the first center coordinates of the digital identifier and the numbers in the digital identifier.

[0082] In this embodiment, the preset target detection model may refer to an algorithm based on deep learning (such as a convolutional neural network), which can automatically identify targets in a digital identifier image and mark information such as their positions and categories. For example, the target detection model can be YOLO (You Only Look Once), Faster R-CNN, SSD (Single Shot MultiBoxDetector), etc. The category information can refer to the category labels of the image regions output by the target detection model. The recognition box is a rectangular box used by the target detection model to mark the positions of targets in the image. In this embodiment, the category information is the digital category represented by the digital identifier region (such as "1", "2", etc.). The recognition box is used to mark the position of the digital identifier in the digital identifier image.

[0083] Specifically, multiple digital identifier images can be used to pre-train the target detection model to obtain a target detection model with a relatively high recognition accuracy. The digital identifier region is extracted, and the center point coordinates are obtained by taking the average of the upper left corner coordinates and the lower right corner coordinates of the recognition box. In addition, the number corresponding to the category information is obtained, so as to determine the first center coordinates and the number of the digital identifier.

[0084] It can be understood that through the target detection model, the digital identifier and the number in the digital identifier image can be quickly and accurately identified, reducing the computing power requirements and providing a basis for further analysis.

[0085] In some embodiments, the first center coordinates of the digital identifier in the digital identifier image and the numbers in the digital identifier can also be determined through other image processing and computer vision techniques. For example, the outer contour of the digital identifier is extracted through edge detection technology, and then the first center coordinates of the digital identifier are determined by the centroid of the outer contour. When the outer contour of the digital identifier has a regular shape, the outer contour of the digital identifier can also be detected by the Hough transform. According to the outer contour information, the first center position of the digital identifier is extracted, and then the numbers in the digital identifier image are recognized through Optical Character Recognition (OCR) technology. In addition, according to the numbers set in the digital identifier, a template library containing these numbers can be made, so as to search and locate the digital identifier in the digital identifier image through the template matching algorithm, and then calculate the centroid or boundary of the matching region, and further determine the first center coordinates and the corresponding numbers.

[0086] S103. Determine the target deviation distance of the first center coordinates relative to the second center coordinates of the digital identifier image.

[0087] Among them, since the digital identifier image obtained by the vision device is usually rectangular, the second center coordinates are the centroid of the rectangle.

[0088] As shown Figure 2 in the figure Figure 2 is a schematic diagram of the digital identifier image provided by the embodiment of the present application. This digital identifier image is the bay position image. In the field of view of the bay position image, the line segment indicated by the arrow is the target deviation distance of the first center coordinate of the digital identifier relative to the second center coordinate of the digital identifier image.

[0089] In this embodiment, the target deviation distance can be obtained by calculating the Euclidean distance between the first center coordinate and the second center coordinate. Similarly, since there are at least two digital identifiers in the digital identifier image, correspondingly, at least two target deviation distances also need to be determined.

[0090] S104. If the number meets the preset continuity requirement, the yard crane is positioned according to the target deviation distance.

[0091] Among them, the preset continuity requirement can refer to the continuity standard specified in the digital identifier. For example, the digital identifier can be set continuously according to Arabic numerals, such as 12345, etc., or can be set to odd numbers (such as 1357, etc.) or even numbers (such as 2468, etc.).

[0092] Exemplarily, if it is not continuous (such as 134, etc.), it indicates that the recognition result is incorrect and the verification fails; if it is continuous, it indicates that the recognition result is correct and the verification passes, and the target deviation distance is returned to complete the positioning of the yard crane.

[0093] It can be understood that through the self-verification judgment of the digital identifier, it can be automatically determined whether there is an error in the recognition process, avoiding misjudgment and improving the recognition accuracy.

[0094] In some embodiments, the digital identifier can also be an English letter, and the continuity requirement satisfied by the English letter is the sequential arrangement in the alphabet, that is, from A to Z.

[0095] In this embodiment, the positioning information of the yard crane includes numbers and target deviation distances. For example, for the digital identifier with the number 2, the target deviation distance of the first center coordinate relative to the second center coordinate is 150.56 pixels, then the corresponding positioning information can be expressed as (2, 150.56); for the digital identifier with the number 3, the target deviation distance of the first center coordinate relative to the second center coordinate is 240.79 pixels, then the corresponding positioning information can be expressed as (3, 240.79). Thus, under specific job position requirements, the specific position of the yard crane can be quickly determined according to the numbers and target deviation distances.

[0096] As Figure 3 shown Figure 3This is a top view of the scene of the quay crane operation provided by the embodiment of the present application. When the quay crane moves along the lane line and moves the container to the yard, if it stops between the digital identifier 1 corresponding to the number 1 and the digital identifier 2 corresponding to the number 2, at this time, the position of the quay crane can be determined according to the number and the target deviation distance.

[0097] Based on the above embodiment, the specific implementation manner of step S104 can be:

[0098] First, determine the scaling ratio of the digital identifier image according to the built-in parameters of the image acquisition device. Then, based on the scaling ratio, perform scaling processing on the target deviation distance to obtain the actual deviation distance corresponding to the target deviation distance. Finally, determine the position of the quay crane according to the actual deviation distance.

[0099] Among them, the image acquisition device is used to acquire the digital identifier image, and can be a hardware device such as a camera, a webcam or other sensors. The image acquisition device is set on the bracket of the quay crane and can continuously track the movement of the quay crane. The built-in parameters refer to the configuration, characteristics and settings of the image acquisition device itself, such as focal length, pixel resolution, shooting angle, sensor type, etc. Through these parameters, the scaling ratio of the digital identifier image can be calculated, so as to scale the target deviation distance in actual application and calculate the actual deviation distance.

[0100] In this embodiment, according to the horizontal field of view (Field of View, FOV) in the built-in parameters of the camera and the distance between the camera and the digital identifier, the horizontal visible width of the camera can be determined: horizontal visible width = 2 × distance between the camera and the digital identifier × tan(horizontal field of view of the camera / 2).

[0101] As Figure 4 shown, Figure 4 This is the main view of the scene of the quay crane operation provided by the embodiment of the present application. According to the target deviation distance and the distance between the camera and the digital identifier, the actual deviation distance between the quay crane and the digital identifier can be calculated and determined by using the principle of similar triangles and the principle of perspective geometry.

[0102] For example, the distance D actual between the camera and the digital identifier is 10 meters; the width D image of the digital identifier in the digital identifier image is 200 pixels; the target deviation distance D target is 50 pixels; calculate the actual deviation distance D = 50 × 10 / 200 = 2.5 meters, that is, the actual deviation distance between the quay crane and the digital identifier is 2.5 meters.

[0103] It can be understood that by calculating the scaling ratio according to the built-in parameters of the image acquisition device, the target deviation distance in the digital identifier image can be converted into the actual deviation distance, ensuring more accurate positioning of the yard crane and avoiding errors caused by the scale difference between the image and the actual physical world.

[0104] In this embodiment, the set spacing between adjacent digital identifiers is less than the target value, and the target value is the sum of the horizontal visible width of the image acquisition device and the horizontal length of the digital identifier.

[0105] Among them, the horizontal visible width can refer to the visual field range that the image acquisition device can capture in the horizontal direction, that is, the maximum width that the device can see. The horizontal length of the digital identifier can refer to the horizontal width of a single digital identifier in the digital identifier image.

[0106] It can be understood that by setting the set spacing between adjacent digital identifiers to be less than the target value, it can ensure that there is enough space for the digital identifiers to be displayed in the digital identifier image, avoiding overlap or confusion, and making the digital identifiers displayed in the digital identifier image clearer and more readable.

[0107] In this embodiment, the digital identifiers are vertically arranged along the lane line, the shape of the digital identifier is a symmetric figure, and the color contrast between the digital area and the non-digital area in the digital identifier is greater than the preset contrast threshold.

[0108] Among them, the shape of the digital identifier can be a symmetric figure such as a rectangle or a circle.

[0109] Exemplarily, assume that the lane line in the operation area of the yard crane is equipped with digital identifiers, the shape (outer contour) of the digital identifier is a rectangle, the digital area of each digital identifier may be dark, such as black or blue, and the non-digital area is light, such as white or yellow. For example, the "1" in the digital identifier may appear dark blue, and the background around the number is bright yellow. Thus, the high contrast between dark blue and bright yellow makes the digital identifier very prominent visually, facilitating visual recognition and ensuring the clear presentation of the digital area.

[0110] It can be understood that by vertically arranging the digital identifiers along the lane line, the problem that the digital identifiers set on the ground are easily blocked can be prevented. In addition, the symmetry of the symmetric figure can simplify the shape analysis and feature extraction process, making it easier to be detected and recognized in image processing, and at the same time making it easier to determine its center coordinates. And the higher contrast enhances the visibility and distinguishability of the digital identifier in the image, which can reduce errors in the recognition process, thereby improving the positioning accuracy. It enables the digital identifier to be clearly visible regardless of how the light changes, ensuring that the key information in the digital identifier can be quickly and accurately extracted.

[0111] The vision-based yard crane positioning method provided by the embodiments of the present application can monitor the position of the yard crane in real time by identifying and calculating the numbers of digital identifiers and the target deviation distance. It can not only achieve precise positioning of the yard crane, but also be applicable to various environments with good adaptability. At the same time, the automated image processing and positioning process reduces manual intervention and improves the simplicity and reliability of the positioning process.

[0112] Figure 5 It is a schematic flow of the vision-based yard crane positioning method provided by the embodiments of the present application. Figure 2 . As Figure 5 shown, on the basis of the Figure 1 embodiment, the method of this embodiment may include the following steps:

[0113] S201. Determine the difference value between the target deviation distance and the standard deviation distance according to the target deviation distance.

[0114] Wherein, the standard deviation distance is the deviation distance between the first center coordinate and the second center coordinate when the yard crane is in the standard operation position. The standard deviation distance is a predefined reference value used to measure the deviation of the yard crane in the standard position.

[0115] In this embodiment, when the corresponding digital identifiers are the same, the target deviation distance is compared with the standard deviation distance to obtain the difference value, so as to determine whether the yard crane deviates from the predetermined position.

[0116] For example, when the number of the digital identifier at the position where the yard crane is located is 3, the standard deviation distance is 152, and the target deviation distance is 158, then the standard operation position can be expressed as (3, 152), the current operation position can be expressed as (3, 155), and the difference value can be expressed as (3, 9), where "3" is the number of the digital identifier and "9" is the difference value corresponding to the digital identifier.

[0117] S202. Determine that the difference value is less than or equal to the preset difference value threshold.

[0118] In this embodiment, the preset difference value threshold can be dynamically adjusted according to the operation position accuracy requirement and the operation environment of the yard crane. If the difference value is less than or equal to the difference value threshold, it means that the deviation is within the allowable range and the positioning of the yard crane is relatively accurate.

[0119] S203. If the difference value is greater than the difference value threshold, it is determined that the yard crane positioning fails, and a positioning failure message is generated. The positioning failure message includes whether the yard crane deviates from the lane line and / or whether the angle of the image acquisition device on the yard crane changes.

[0120] In this embodiment, when the yard crane deviates from the lane line, it indicates that there is a deviation during the movement of the yard crane and the deviation is serious, then the positioning function of the yard crane may fail. When the angle of the image acquisition device on the yard crane changes, it will affect the shooting perspective and the quality of the digital identifier image, so it is also possible that the angle of the image acquisition device is different from the set angle.

[0121] As an example, the above two situations may occur simultaneously. For example, due to external forces (such as high-speed driving, equipment vibration, external obstacle influence, etc.), the yard crane deviates from the lane line. At the same time, when the image acquisition device is affected by vibration or external force, its angle may also change. The simultaneous occurrence of these two situations may seriously affect the positioning accuracy and the quality of image acquisition, resulting in the system judging the positioning to fail.

[0122] As another example, one of the above two situations may occur alternatively. For example, if the yard crane deviates from its position during movement due to road conditions, improper driver control, attitude adjustment, etc. In this case, the angle of the image acquisition device usually does not change significantly immediately.

[0123] The vision-based yard crane positioning method provided by the embodiment of the present application monitors the difference between the target deviation distance and the standard deviation distance, and judges whether the difference is within an acceptable range. If the difference value is greater than the threshold, the system will consider that the yard crane positioning fails and generate a positioning failure message, so as to help further analyze the reason for the positioning failure.

[0124] Figure 6 It is a schematic structural diagram of a vision-based yard crane positioning device provided by an embodiment of the present application. As Figure 6 shown, the vision-based yard crane positioning device 30 provided in this embodiment includes:

[0125] An acquisition module 301, configured to acquire a digital identifier image of the working position of the yard crane, where the digital identifier image includes at least two digital identifiers, and the digital identifiers are arranged along the lane line corresponding to the working position;

[0126] A first determination module 302, configured to determine the first central coordinates of the digital identifiers in the digital identifier image and the numbers in the digital identifiers;

[0127] A second determination module 303, configured to determine the target deviation distance of the first central coordinates relative to the second central coordinates of the digital identifier image;

[0128] A positioning module 304, configured to position the yard crane according to the target deviation distance if the numbers meet the preset continuity requirements.

[0129] In a possible implementation manner, the positioning module 304 is further configured to:

[0130] Determine the scaling ratio of the digital identifier image according to the built-in parameters of the image acquisition device, where the image acquisition device is used to acquire the digital identifier image, and the image acquisition device is arranged on the bracket of the gantry crane;

[0131] Based on the scaling ratio, perform scaling processing on the target deviation distance to obtain the actual deviation distance corresponding to the target deviation distance;

[0132] Determine the position of the gantry crane according to the actual deviation distance.

[0133] In a possible implementation manner, the second determination module 303 is further configured to:

[0134] Determine the difference value between the target deviation distance and the standard deviation distance according to the target deviation distance, where the standard deviation distance is the deviation distance between the first center coordinate and the second center coordinate when the gantry crane is at the standard operation position;

[0135] Determine that the difference value is less than or equal to a preset difference value threshold.

[0136] In a possible implementation manner, the second determination module 303 is further configured to:

[0137] If the difference value is greater than the difference value threshold, then determine that the gantry crane positioning fails, and generate a positioning failure message, where the positioning failure message includes whether the gantry crane deviates from the lane line and / or whether the angle of the image acquisition device on the gantry crane changes.

[0138] In a possible implementation manner, the first determination module 302 is specifically configured to:

[0139] Based on a preset target detection model, identify the digital identifier image to obtain the digital identifier area in the digital identifier image and the category information corresponding to the digital identifier area;

[0140] Determine the center point coordinates of the recognition box corresponding to the digital identifier area according to the digital identifier area;

[0141] Determine the number corresponding to the category information according to the category information corresponding to the digital identifier area;

[0142] Determine the first center coordinate of the digital identifier and the number in the digital identifier according to the center point coordinates of the recognition box and the number corresponding to the category information.

[0143] The vision-based gantry crane positioning device provided in this embodiment can execute the method provided in the above method embodiment, and its implementation principle and technical effect are similar, which will not be elaborated here in this embodiment.

[0144] Figure 7 This is a schematic structural diagram of the vision-based gantry crane positioning system provided in the embodiments of the present application. AsFigure 7 As shown in the figure, the vision-based yard crane positioning system 40 provided in this embodiment includes:

[0145] An image acquisition device 401, configured to acquire a digital identifier image of the working position of the yard crane;

[0146] An electronic device 402, configured to implement various possible implementation manners as described above.

[0147] The vision-based yard crane positioning system provided in this embodiment can execute the method provided in the above method embodiment, and its implementation principle and technical effect are similar, which will not be elaborated here in this embodiment.

[0148] Figure 8 It is a schematic structural diagram of the electronic device provided in this application. As Figure 8 shown, the electronic device 50 provided in this embodiment includes: at least one processor 501 and a memory 502. Optionally, the device 50 further includes a communication component 503. Among them, the processor 501, the memory 502, and the communication component 503 are connected through a bus 504.

[0149] In a specific implementation process, at least one processor 501 executes the computer execution instructions stored in the memory 502, so that at least one processor 501 executes the above method.

[0150] For the specific implementation process of the processor 501, reference can be made to the above method embodiment, and its implementation principle and technical effect are similar, which will not be elaborated here in this embodiment.

[0151] In the above embodiment, it should be understood that the processor may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the invention can be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0152] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0153] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the buses in the drawings of this application are not limited to only one bus or one type of bus.

[0154] This application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0155] This application also provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the processor executes the computer-executable instructions, the above method is implemented.

[0156] The above-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disc. The readable storage medium can be any available medium accessible by a general-purpose or special-purpose computer.

[0157] An exemplary readable storage medium is coupled to the processor, enabling the processor to read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an Application Specific Integrated Circuit (ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in a device.

[0158] The division of units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces, and the indirect coupling or communication connection of devices or units can be in electrical, mechanical, or other forms.

[0159] The unit described as a separation component may or may not be physically separated. The component displayed as a unit may or may not be a physical unit, that is, it may be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0160] In addition, each functional unit in various embodiments of the present invention may be integrated into a processing unit, may exist separately as individual physical units, or two or more units may be integrated into one unit.

[0161] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or part of the technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0162] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above method embodiments; and the foregoing storage medium includes: various media such as ROM, RAM, magnetic disks, or optical discs that can store program codes.

[0163] Finally, it should be noted that: after considering the specification and practicing the invention disclosed herein, those skilled in the art will easily think of other implementation schemes of the present invention. The present invention aims to cover any variations, uses, or adaptive changes of the present invention. These variations, uses, or adaptive changes follow the general principles of the present invention and include common general knowledge or conventional technical means in the technical field not disclosed in the present invention. It is not limited to the precise structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. A field bridge positioning method based on vision, characterized in that: include: Acquire a digital identifier image of an operation position of a field crane, wherein the digital identifier image includes at least two digital identifiers, and the digital identifiers are arranged along a lane line corresponding to the operation position; determining first center coordinates of a digital identifier in the digital identifier image and a number in the digital identifier; determining a target deviation distance of the first center coordinate relative to a second center coordinate of the digital identifier image; If the number meets the preset continuity requirement, the field bridge is positioned according to the target deviation distance.

2. The method according to claim 1, characterized in that Positioning the field bridge according to the target deviation distance includes: Determining a scaling ratio of the digital identifier image according to built-in parameters of an image acquisition device, the image acquisition device being used to acquire the digital identifier image, the image acquisition device being disposed on a bracket of the field bridge; Based on the scaling ratio, scaling the target deviation distance to obtain an actual deviation distance corresponding to the target deviation distance; The position of the field bridge is determined according to the actual deviation distance.

3. The method according to claim 2, characterized in that The setting interval between adjacent digital identifiers is smaller than a target value, and the target value is the sum of a horizontal visible width of the image acquisition device and a horizontal length of the digital identifier.

4. The method according to any one of claims 1 to 3, characterized in that The digital identifier is vertically arranged along the lane line, the shape of the digital identifier is a symmetrical figure, and the color contrast between the digital area and the non-digital area in the digital identifier is greater than a preset contrast threshold.

5. The method according to any one of claims 1 to 3, characterized in that Before positioning the field bridge according to the number and the target deviation distance, the method further includes: Determine, according to the target deviation distance, a difference value between the target deviation distance and a standard deviation distance, wherein the standard deviation distance is a deviation distance between the first center coordinate and the second center coordinate when the field crane is in the standard operating position; It is determined that the difference value is less than or equal to a preset difference value threshold.

6. The method according to claim 5, characterized in that The method further comprises: If the difference value is greater than the difference value threshold, it is determined that the field bridge positioning has failed, and positioning failure information is generated. The positioning failure information includes whether the field bridge deviates from the lane line and / or whether the angle of the image acquisition device on the field bridge has changed.

7. The method according to any one of claims 1 to 3, characterized in that The determining of the first center coordinates of the digital identifier in the digital identifier image and the number in the digital identifier comprises: Based on a preset target detection model, the digital identifier image is identified to obtain a digital identifier region in the digital identifier image and category information corresponding to the digital identifier region; According to the digital identifier area, determining the coordinates of the center point of the identification box corresponding to the digital identifier area; Determining, according to the category information corresponding to the digital identifier area, a number corresponding to the category information; According to the center point coordinates of the identification frame and the numbers corresponding to the category information, the first center coordinates of the digital identifier and the numbers in the digital identifier are determined.

8. A field bridge positioning device based on vision, characterized in that: include: An acquisition module, used for acquiring a digital identifier image of an operation position of a field crane, wherein the digital identifier image includes at least two digital identifiers, and the digital identifiers are arranged along a lane line corresponding to the operation position; A first determination module, configured to determine a first center coordinate of a digital identifier in the digital identifier image and a number in the digital identifier; a second determination module, configured to determine a target deviation distance of the first center coordinate relative to a second center coordinate of the digital identifier image; A positioning module is used to position the field bridge according to the target deviation distance if the number meets the preset continuity requirement.

9. A field bridge positioning system based on vision, characterized in that: include: An image acquisition device, used to acquire a digital identifier image of the operating position of the field crane; An electronic device, used to implement the method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 7.

Images