Container Recognition Method, System, Device and Storage Medium Based on Spatial Scanning

Abstract

The present invention provides a container recognition method, system, device and storage medium based on spatial scanning. The method includes the following steps: collecting spatial scanning data of the crane hoisting a container through a spreader positioning device to establish a training set, and obtaining a prediction model for judging whether the crane hoists a container. The spatial scanning data includes at least the position information of the spreader and the position information of the container; detecting the spatial scanning data of the current state of the crane by using the spreader positioning device; when it is judged by the prediction model that the crane has hoisted a container, obtaining the rotation angle information and focal length information between the target surface of the container and the container recognition component according to the position information of the container in the spatial scanning data at this time; rotating the image acquisition device according to the rotation angle information and taking an image according to the focal length information; performing graphic and text recognition on the taken image to obtain the recognition code of the container. The present invention can recognize the hoisted container and realize intelligent container tallying.

Description

Background Art

[0002] At present, tallying of outer wheels is an essential link in the port container loading and unloading operation. For a long time, tallying operations have been carried out by personnel on the operation site, with high work intensity and great danger. With the development of camera technology, especially the technological innovation of artificial intelligence in image detection, it is possible to withdraw and reduce the number of on-site tally clerks. Through more accurate image recognition technology, replacing manual tally clerks to achieve remote intelligent recognition is an inevitable trend in the development of port intelligent tallying.

[0003] Compared with the intelligent quay crane for large-scale operations, the operation mode of fixed lifting and horizontal transfer has a very long movement stroke. Due to the random rotation of the jib and the parking position of horizontal transport vehicles in the loading and unloading operations of portal cranes and fixed jib cranes, the focal distance changes greatly, resulting in the difficulty of the container information necessary for their intelligent tallying to appear in a fixed picture. Moreover, due to the concentrated stacking of a large number of containers, information such as the identification codes of multiple containers often appears in one picture, reducing the accuracy of image recognition in identifying the identification codes of the containers to be operated. Separately through multiple groups of fixed video acquisition positions, only the functions of partial operation intelligent recognition can be satisfied. For common multi-lane operations, horizontal operations cannot ensure that the container and vehicle information to be recognized appears in the picture. Therefore, for a long time, the intelligent tallying of portal crane and fixed jib crane operations cannot meet the requirements of intelligent recognition and personnel reduction.

[0004] Therefore, the present invention provides a container recognition method, system, device and storage medium based on spatial scanning. Summary of the Invention

[0005] Aiming at the problems in the prior art, the purpose of the present invention is to provide a container recognition method, system, device and storage medium based on spatial scanning, which overcomes the difficulties of the prior art and can realize the follow-up tracking recognition of the container being lifted through the integration of a spreader positioning sensor to guide a follow-up image sensor, so as to realize the intelligent tallying of container operations.

[0006] An embodiment of the present invention provides a container recognition method based on spatial scanning, which uses at least one container recognition component integrated with a spreader positioning device and an image acquisition device, and includes the following steps:

[0007] S110. Establish a training set by collecting spatial scanning data of the crane lifting the container through the spreader positioning device, and obtain a prediction model for judging whether the crane is lifting a container. The spatial scanning data at least includes the position information of the spreader and the position information of the container;

[0008] S120. Detect the spatial scanning data of the current state of the crane by using the spreader positioning device;

[0009] S130. When it is determined by the prediction model that the crane has hoisted a container, according to the position information of the container in the spatial scan data at this time, obtain the rotation angle information and focal length information of a target surface of the container and the container identification component;

[0010] S140. The image acquisition device rotates according to the rotation angle information and takes an image according to the focal length information;

[0011] S150. Perform graphic recognition on the captured image to obtain the recognition code on the container surface.

[0012] Preferably, the image acquisition device has an image sensor, a steering pan-tilt head and a focusing module. The steering pan-tilt head rotates the image sensor of the image acquisition device to align with the container according to the rotation angle information. After the focusing module adjusts the focal length of the image acquisition device according to the focal length information, an image with at least a partial outer surface of the container is taken.

[0013] Preferably, the spreader positioning device is a point cloud sensor, the spatial scan data is a three-dimensional point cloud data set, the position information of the spreader is a three-dimensional point cloud set on the surface of the spreader, and the position information of the container is a three-dimensional point cloud set on multiple surfaces of the container.

[0014] Preferably, the spreader positioning device is one or a combination of a 2D laser sensor, a 3D laser sensor, an infrared sensor, a millimeter wave radar, a monocular or binocular vision sensor.

[0015] Preferably, the prediction model determines that the crane has hoisted a container according to the fact that the container is directly below the crane and the container and the crane perform synchronous movement in the horizontal direction.

[0016] Preferably, the point cloud sensor is a laser sensor. The laser sensor performs circumferential scanning based on the crane, generates laser points on the surfaces of the spreader and the container, takes the laser sensor as the coordinate origin, and obtains the spatial coordinates of the laser points in the laser coordinate system of the laser sensor. A training set is established according to the set of the spatial coordinates of the laser points in the first state when the crane hoists the container and the second state when the crane does not hoist the container.

[0017] Preferably, obtain the focal length information according to the distance between the spatial coordinates of the center point of the target surface of the container and the spatial coordinates of the container identification component, and obtain the rotation angle information according to the direction between the spatial coordinates of the center point of the target surface of the container and the spatial coordinates of the container identification component.

[0018] Preferably, spatial position information of multiple surfaces of the container is generated based on the three-dimensional point cloud set of the container surface.

[0019] When the angle between the square surface at one end of the container and the container identification component is within the first preset threshold, the spatial coordinates of the midpoint of the square surface are used as the target coordinates; or, when the angle between a rectangular surface of the container and the container identification component is within the second preset threshold, the spatial coordinates of the midpoint of a preset area in the rectangular surface are used as the target coordinates.

[0020] Focal length information and rotation angle information are obtained based on the target coordinates.

[0021] Preferably, the container identification component is fixed on the surface of the crane.

[0022] Preferably, the crane is a gantry crane, and two container identification components are arranged on the gantry crane, one is arranged below the cockpit, and the other is arranged at the land side leg of the gantry crane.

[0023] Preferably, the target placement position preset for the container is obtained according to the identification code of the container, and the crane hoists the container to the target placement position.

[0024] An embodiment of the present invention further provides a container identification system based on spatial scanning for implementing the above-mentioned container identification method based on spatial scanning. The container identification system based on spatial scanning includes:

[0025] A container identification component, the container identification component includes a spreader positioning device and an image acquisition device. The spreader positioning device is one or a combination of a 2D laser sensor, a 3D laser sensor, an infrared sensor, a millimeter wave radar, a monocular or binocular vision sensor. The image acquisition device has an image sensor, a steering pan-tilt head and a focusing module, and the steering pan-tilt head and the focusing module are respectively connected to the container identification component.

[0026] Preferably, the container identification component is fixed on the surface of the crane, and the container identification component integrates a spreader positioning device and an image acquisition device for detecting the position of the container.

[0027] Preferably, the container identification component includes:

[0028] A first bracket, one end of the first bracket is fixed on the surface of the crane;

[0029] A duckbill bracket is arranged on the upper surface of the other end of the first bracket;

[0030] A second bracket, the lower surface of the second bracket being connected to the duckbill bracket;

[0031] A spreader positioning device, disposed on the upper surface of the second bracket; and

[0032] An image acquisition device, disposed on the lower surface of the other end of the first bracket, the steering pan-tilt and focus module of the image acquisition device being connected to the spreader positioning device through a data cable.

[0033] Preferably, the crane is a gantry crane, and the two container identification components are disposed on the gantry crane, one being disposed below the cab and the other being disposed at the land side leg of the gantry crane.

[0034] Preferably, the crane is a fixed jib crane.

[0035] An embodiment of the present invention further provides a container identification device based on spatial scanning, including:

[0036] A processor;

[0037] A memory, in which executable instructions of the processor are stored;

[0038] Wherein, the processor is configured to execute the steps of the above-mentioned container identification method based on spatial scanning by executing the executable instructions.

[0039] An embodiment of the present invention further provides a computer-readable storage medium for storing a program, and when the program is executed, the steps of the above-mentioned container identification method based on spatial scanning are implemented.

[0040] The container identification method, system, device and storage medium based on spatial scanning of the present invention can realize the follow-up tracking identification of the container to be lifted through the fusion of the spreader positioning sensor and the guided follow-up image sensor, so as to realize the intelligent tallying of container operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, objects and advantages of the present invention will become more obvious.

[0042] Figure 1 is a flowchart of the container identification method based on spatial scanning of the present invention.

[0043] Figure 2 is a schematic diagram of the scenario for implementing the container identification method based on spatial scanning of the present invention.

[0044] Figure 3 is a schematic structural diagram of the container identification component in the container identification system based on spatial scanning of the present invention.

[0045] Figures 4 to 7 It is a schematic diagram of the implementation process of the container identification method based on spatial scanning of the present invention.

[0046] Figure 8 It is a schematic structural diagram of the container identification device based on spatial scanning of the present invention. And

[0047] Figure 9 It is a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present invention.

[0048] Reference numerals

[0049] 1 Container identification component

[0050] 11 First bracket

[0051] 12 Duckbill bracket

[0052] 14 Second bracket

[0053] 15 Spreaders positioning device

[0054] 16 Image acquisition device

[0055] 2 Crane

[0056] 21 Spreaders

[0057] 22 Boom

[0058] 3 Container

[0059] 4 Container

[0060] 5 Container identification component Detailed implementation manners

[0061] Now, example embodiments will be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. Like reference numerals in the figures denote like or similar structures, and thus their repetitive description will be omitted.

[0062] Figure 1 It is a flowchart of the container identification method based on spatial scanning of the present invention. Figure 2 It is a schematic diagram of implementing the container identification method based on spatial scanning of the present invention. An embodiment of the present invention provides a container identification method based on spatial scanning, which employs at least one container identification component 1 integrated with a spreaders positioning device 15 and an image acquisition device 16, and includes the following steps:

[0063] S110. Establish a training set by collecting spatial scan data of the crane 2 hoisting the container 4 through the spreader positioning device 15, and obtain a prediction model for judging whether the crane 2 hoists the container 4. The spatial scan data includes at least the position information of the spreader 21 and the position information of the container 4.

[0064] S120. Detect the spatial scan data of the current state of the crane 2 by using the spreader positioning device 15.

[0065] S130. When it is judged by the prediction model that the crane 2 has hoisted the container 4, obtain the rotation angle information and focal length information of a target surface of the container 4 and the container identification component 1 according to the position information of the container 4 in the spatial scan data at this time.

[0066] S140. The image acquisition device 16 rotates according to the rotation angle information and takes an image according to the focal length information.

[0067] S150. Perform graphic and text recognition on the captured image to obtain the recognition code on the surface of the container 4.

[0068] S160. Obtain the preset target placement position of the container 4 according to the recognition code of the container 4, and the crane 2 hoists the container 4 to the target placement position.

[0069] In view of the characteristic that the portal crane cannot standardize the starting position of each operation, the present invention adopts a multi-sensor fusion to real-time identify the working spreader and the grasping and releasing state of the container, and track the real-time position of the spreader.

[0070] In a preferred embodiment, the image acquisition device 16 has an image sensor, a steering pan-tilt and a focusing module. The steering pan-tilt rotates the image sensor of the image acquisition device 16 according to the rotation angle information to align with the container 4, and the focusing module adjusts the focal length of the image acquisition device 16 according to the focal length information and then takes an image with at least a partial outer surface of the container 4.

[0071] Due to the randomness of the loading and unloading operations of the portal crane and the large movement range of the spreader, the present invention uses a prediction model obtained by establishing a training set through the cooperation of the spreader positioning device 15 and the spatial scan data of the spreader's historical hoisting to determine whether the spreader hoists the container. When the spreader hoists the container, the spatial position of the container is determined according to the spatial scan data obtained by the spreader positioning device 15, so that the image acquisition device 16 can accurately align with the container through angle rotation, and the focal length obtained from the spatial position enables the image acquisition device 16 to accurately focus on the container, thereby obtaining the clearest container image and greatly improving the accuracy of subsequent container identification. It also effectively prevents the interference of other containers 3 stacked on the ground to the container identification.

[0072] In a preferred embodiment, the spreader positioning device 15 is a point cloud sensor, the spatial scan data is a three-dimensional point cloud data set, the position information of the spreader 21 is a three-dimensional point cloud set of the spreader 21, and the position information of the container 4 is a three-dimensional point cloud set of multiple surfaces of the container 4, but not limited thereto. The three-dimensional point cloud data set in the present invention is a point cloud obtained by using an infrared image acquisition device, a cumulative scanning method of a two-dimensional laser sensor, a three-dimensional laser scanner or a photogrammetric scanner. The number of points is relatively large and relatively dense, which is called a dense point cloud. A point cloud is a massive set of points on the target surface characteristics. The point cloud obtained according to the laser measurement principle includes three-dimensional coordinates (XYZ) and laser reflection intensity (Intensity). The point cloud obtained according to the photogrammetry principle includes three-dimensional coordinates (XYZ) and color information (RGB). The point cloud obtained by combining the laser measurement and photogrammetry principles includes three-dimensional coordinates (XYZ) and color information (RGB). After obtaining the spatial coordinates of each sampling point on the object surface, a set of points is obtained, which is called a "point cloud" (Point Cloud).

[0073] In a preferred embodiment, the spreader positioning device 15 is one or a combination of a 2D laser sensor, a 3D laser sensor, an infrared sensor, a millimeter wave radar, a monocular or binocular vision sensor. The principle of the 3D laser sensor used in the present invention is that when a laser beam irradiates the surface of an object, the reflected laser will carry information such as azimuth and distance. If the laser beam is scanned along a certain trajectory, the information of the reflected laser points will be recorded while scanning. Since the scanning is extremely fine, a large number of laser points can be obtained, and thus a laser point cloud can be formed.

[0074] In a preferred embodiment, the prediction model determines that the crane 2 hoists the container 4 based on the fact that the container 4 is directly below the crane 2 and the container 4 and the crane 2 move synchronously in the horizontal direction. Since the container 4 hoisted by the crane 2 will be directly below the crane 2 under the influence of gravity and move synchronously with the crane 2 in the horizontal direction, the prediction model can accurately identify the hoisted container 4 from a large number of containers (which may also include the containers 3 stacked on the ground) according to the spatial position based on the above characteristics.

[0075] In a preferred embodiment, the point cloud sensor is a laser sensor. The laser sensor performs circumferential scanning based on the crane 2, generates laser points on the surfaces of the spreader 21 and the container 4, and takes the laser sensor as the coordinate origin to obtain the spatial coordinates of the laser points in the laser coordinate system of the laser sensor. A training set is established based on the set of the spatial coordinates of the laser points in the first state where the crane 2 hoists the container 4 and the second state where the crane 2 does not hoist the container 4.

[0076] In a preferred embodiment, the focal length information is obtained based on the distance between the spatial coordinates of the center point of the target surface of the container 4 and the spatial coordinates of the container identification component 1, and the rotation angle information is obtained based on the direction between the spatial coordinates of the center point of the target surface of the container 4 and the spatial coordinates of the container identification component 1.

[0077] In a preferred embodiment, the spatial position information of multiple surfaces of the container 4 is generated based on the three-dimensional point cloud set of the container 4. Since at most three surfaces of the container 4 are irradiated by the laser sensor at the same time, and the other three surfaces of the container 4 are not irradiated by the laser sensor, there may be at least one and at most three surfaces of the rectangular container 4 in the three-dimensional point cloud set of the container 4. Considering that there are identification codes provided at multiple locations on the whole body of the container 4, but not all surfaces are in the best shooting position (the surface with the identification code is vertically facing the container identification component 1), therefore, the angle difference between the two needs to be considered. When the angle between the square surface at one end of the container 4 and the container identification component 1 is within the first preset threshold, the first preset threshold can be 45°, then the spatial coordinates of the midpoint of the square surface are used as the target coordinates. At this time, the square surface is more directly facing the container identification component 1, and the taken picture is easier to identify the identification code. Or, when the angle between a rectangular surface of the container 4 and the container identification component 1 is within the second preset threshold, the second preset threshold can be 45°, then the spatial coordinates of the midpoint of the preset area in the rectangular surface are used as the target coordinates (since the identification code is usually marked at the upper left position of the rectangular side surface of the container, the upper left rectangular area of the rectangular side surface of the container can be used as the preset area, but not limited thereto). At this time, the rectangular surface is more directly facing the container identification component 1, and the taken picture is easier to identify the identification code. Then, the focal length information and the rotation angle information are obtained based on the target coordinates, and the container picture is taken.

[0078] In a preferred embodiment, the container identification component 1 is fixed on the surface of the crane 2, but not limited thereto.

[0079] In a preferred embodiment, the crane 2 is a gantry crane, and two container identification components are arranged on the gantry crane. One container identification component 1 is arranged below the cab, and the other container identification component 5 can be arranged at the land side leg of the gantry crane, but not limited thereto.

[0080] In a preferred embodiment, the crane 2 is a fixed jib crane, but not limited thereto.

[0081] This system can consist of a main identification system (for example: Figure 2 the container identification component 1 in Figure 2It consists of a container identification component 5), and multiple sensors are linked to complete the identification. Compared with other devices and systems that use only visual recognition, the present invention has better recognition effects for multiple lanes under the gantry crane, random operation starting points, and lateral operations. The sensors in the multiple sensor fusion system include, but are not limited to, 2D laser detection devices, 3D laser detectors, monocular or multiocular vision cameras, millimeter wave radars, etc., which can be active and passive detection devices or apparatuses for target detection.

[0082] In this system, the main recognition system is composed of a positioning sensor and an identification sensor, and is connected by an integrated bracket to ensure the consistency of the observation coordinates. This design determines a unified perception coordinate system through a unified mechanical support structure, avoiding the joint calibration process required in common multi-sensor systems. While ensuring the recognition accuracy, it minimizes the loss of the engineering implementation cycle caused by on-site calibration. In actual engineering installation, the main recognition system is installed below the driver's cab of the gantry crane and moves synchronously with the direction of the driver's cab and the jib to ensure the forward field of view for recognition. In addition, to ensure that the recognition accuracy can meet the usage requirements, a supplementary recognition sensing system is installed on the far-end land side leg of the portal crane for supplementary recognition. The supplementary recognition system is linked and controlled according to the main recognition system to achieve the detection and recognition of the target by angle.

[0083] In the recognition process of this system, a follow-up tracking technology is adopted. For most portal crane scenarios, all recognition can be completed through the main recognition system. The positioning sensor in the main recognition system follows and locks the position of the spreader through detection signals (including but not limited to lasers, camera images, millimeter wave radars, infrared detections), and calculates the positional relationship between each recognition surface of the container and the main and auxiliary recognition systems.

[0084] The target positioning calculation process is as follows:

[0085] Where:

[0086] (x i , y i , z i ) represents the coordinate position of a target in space, and the coordinate origin (0, 0, 0) is the physical center of the positioning sensor.

[0087] r i represents the target detection data information of the positioning sensor.

[0088] is the total detection data intensity information in one frame of positioning sensor detection.

[0089] By performing static registration on two frames of sensor detection data, the rotation matrix R and the translation matrix T of the two frames of static data are calculated. The specific algorithm is as follows:

[0090] Iterative closest point algorithm for multi-frame detection data matching. Assume that at a certain moment, the positioning sensor receives a set of detection data P = {p1, p2,..., p n}, after rotation and translation, a second set of detection data Q = {q1, q2,..., q n} is obtained. Through the iterative closest point method, the matching pairs corresponding to the same point in three-dimensional space in P and Q can be obtained.

[0091] Assume that the rotation of the positioning sensor is R and the translation vector is t. Then the formula for converting a point in the P coordinate system to the Q coordinate system is

[0092] q i = R·p i + t

[0093] And the objective function of the positioning detection is

[0094]

[0095] By defining the density cores of the two sets of detection data before and after as

[0096]

[0097] Since in the last term

[0098]

[0099] Let p′ i = p i - μ p and q′ i = q i - μ q Then the objective function can be simplified to:

[0100]

[0101] Let R * , t * be the optimal solution. The optimization problem can be divided into two steps:

[0102]

[0103] t * = μ q - R·μ p

[0104] For step 1, expanding it gives

[0105]

[0106] Let Through SVD decomposition, we have

[0107] W = USV T

[0108] Corresponding to the unique combination of U and V, the corresponding

[0109] R * = UV T

[0110] t * = μ q - R·μ p

[0111] After calculating the rotation matrix between the two sets of data before and after, considering that the horizontal rotation during the rotation of the portal crane can be ignored, so we can get

[0112] cosα = R * [0, 0]

[0113] sinα = R * [1, 0]

[0114] where α ∈ [-180°, 180°]

[0115] Through α, the distance of the portal crane rotation between the two sets of detection data before and after can be obtained. After continuous multi-frame calculations, the overall rotation angle of the portal crane relative to the reference frame during the operation can be obtained, thus helping to determine the operation occurrence location.

[0116] According to the real-time position and angle fed back by the recognition result, the main and auxiliary recognition systems of two fixed positions of the portal crane are called for tracking and scanning to achieve precise real-time follow-up of the container. Then, through artificial intelligence recognition technology, the container type, container number, operating vehicle, and operating lane are accurately and continuously recognized, and the data is synchronized to the operation system terminal. Since the recognition process adopts a tracking mechanism, the recognition can be completed whether it is a container loading or unloading operation. The recognition window is not limited to when the container is on the ground bracket, in the air movement, or during the ship's hold loading and unloading process.

[0117] The auxiliary recognition system is a supplement to the main recognition system. The auxiliary recognition system is installed at the land side portal leg of the crane and adopts a fixed position without rotating with the boom. Under the control of the main recognition system, it realizes the tracking and recognition of the container and operation information. When the main recognition system cannot complete the recognition due to occlusion or poor angle, it provides supplementary recognition ability. The auxiliary recognition system calculates its own observation angle based on the target data provided by the positioning sensor of the main recognition system based on the relative position relationship to complete the rotation control of the camera.

[0118] Compared with other gantry crane intelligent tallying systems, the present invention can complete the identification process only by using an integrated identification system, without a large number of pan-tilt cameras, greatly reducing the construction difficulty. Moreover, it avoids the implementation differences caused by the crane's own model, height, and size.

[0119] The main identification system in the present invention adopts an integrated design and uses a multi-sensor fusion identification method to achieve follow-up tracking identification. The positioning sensor in the main identification system can provide accurate position information of the spreader, container, and vehicle, guiding the identification sensors in the main and auxiliary identification systems to complete accurate identification. A better identification effect is obtained through complementary detection by multiple sensors.

[0120] The main identification system in the present invention adopts a standard support structure. By means of a fixed structure, the problem that multiple sensors need to be pre-calibrated is solved, greatly improving the product's delivery ability when leaving the factory and meeting the application conditions of the scenario.

[0121] The present invention also realizes multi-view follow-up through an auxiliary identification system, fuses the identification results from different angles to obtain more accurate identification accuracy, and at the same time makes up for the problem that it is impossible to identify in advance due to occlusion in special working conditions such as double-container operation on the ground, horizontal operation, empty and heavy loading and unloading operation, and in-cabin operation.

[0122] Figure 3 It is a schematic structural diagram of a container identification component in the container identification system based on spatial scanning of the present invention. As shown in Figure 3, an embodiment of the present invention also provides a container identification system based on spatial scanning for implementing the above-mentioned container identification method based on spatial scanning. The container identification system based on spatial scanning includes: a container identification component 1. The container identification component integrates a spreader positioning device 15 and an image acquisition device 16, but is not limited thereto. The spreader positioning device 15 is one or a combination of a 2D laser sensor, a 3D laser sensor, an infrared sensor, a millimeter-wave radar, a monocular or binocular vision sensor. The image acquisition device 16 has an image sensor, a steering pan-tilt, and a focusing module. The steering pan-tilt and the focusing module are respectively connected to the container identification component. The steering pan-tilt can drive the image sensor and the focusing module to rotate based on the crane, so that the image sensor can accurately align with the container.

[0123] In a variant, the spreader positioning device 15 and the image acquisition device 16 can also be separately arranged, for example, at different heights of the crane 2, etc., but is not limited thereto.

[0124] In a preferred embodiment, the container identification component is fixed on the surface of the crane 2.

[0125] In a preferred embodiment, the container identification component includes:

[0126] A first support 11, one end of the first support 11 is fixed to the surface of the crane 2.

[0127] A duckbill support 12, arranged on the upper surface of the other end of the first support 11.

[0128] A second support 14, the lower surface of the second support 14 is connected to the duckbill support 12.

[0129] A spreader positioning device 15, arranged on the upper surface of the second support 14. And

[0130] An image acquisition device 16, arranged on the lower surface of the other end of the first support 11, the steering pan-tilt and focus module of the image acquisition device 16 are connected to the spreader positioning device 15 through a data cable.

[0131] In a preferred embodiment, the crane 2 is a gantry crane, two container identification components are arranged on the gantry crane, one is arranged below the cockpit, and the other is arranged at the land side leg of the gantry crane.

[0132] In a preferred embodiment, the image acquisition device 16 has an image sensor, a steering pan-tilt and a focus module. The steering pan-tilt rotates the image sensor of the image acquisition device 16 according to the rotation angle information to align with the container 4. After the focus module adjusts the focal length of the image acquisition device 16 according to the focal length information, an image with at least a partial outer surface of the container 4 is taken.

[0133] In a preferred embodiment, the focal length information is obtained according to the distance between the spatial coordinates of the center point of the target surface of the container 4 and the spatial coordinates of the container identification component 1, and the rotation angle information is obtained according to the direction between the spatial coordinates of the center point of the target surface of the container 4 and the spatial coordinates of the container identification component 1.

[0134] In a preferred embodiment, the spreader positioning device 15 is a point cloud sensor, the spatial scan data is a three-dimensional point cloud data set, the position information of the spreader 21 is a three-dimensional point cloud set of the spreader 21, and the position information of the container 4 is a three-dimensional point cloud set of multiple surfaces of the container 4, but not limited thereto. The three-dimensional point cloud data set in the present invention is a point cloud obtained by using a three-dimensional laser scanner or a photogrammetric scanner, with a relatively large and dense number of points, called a dense point cloud. A point cloud is a massive set of points on the target surface characteristics. The point cloud obtained according to the laser measurement principle includes three-dimensional coordinates (XYZ) and laser reflection intensity (Intensity). The point cloud obtained according to the photogrammetry principle includes three-dimensional coordinates (XYZ) and color information (RGB). The point cloud obtained by combining the laser measurement and photogrammetry principles includes three-dimensional coordinates (XYZ), laser reflection intensity (Intensity), and color information (RGB). After obtaining the spatial coordinates of each sampling point on the object surface, a set of points is obtained, which is called a "point cloud" (Point Cloud).

[0135] In a preferred embodiment, the spreader positioning device 15 is one or a combination of a 2D laser sensor, a 3D laser sensor, an infrared sensor, a millimeter wave radar, a monocular or binocular vision sensor. The principle of the 3D laser sensor used in the present invention is that when a laser beam irradiates the surface of an object, the reflected laser beam will carry information such as azimuth and distance. If the laser beam is scanned along a certain trajectory, the information of the reflected laser points will be recorded while scanning. Since the scanning is extremely fine, a large number of laser points can be obtained, and thus a laser point cloud can be formed.

[0136] Figures 4 to 7 It is a schematic diagram of the implementation state of the container identification method based on spatial scanning of the present invention. As Figure 4As shown in the figure, the usage process of the container recognition system based on spatial scanning of the present invention is as follows: The container recognition component 1 is fixed on the surface of the crane 2. The crane 2 is a gantry crane. A spreader 21 is suspended at the outer end of the boom 22 of the crane 2 for lifting the container 3 on the ground. Two container recognition components are provided on the gantry crane. One container recognition component 1 is arranged below the cockpit, and the other container recognition component 5 can be arranged at the land side leg of the gantry crane, but this is not limited thereto. The container recognition component 1 integrates a spreader positioning device 15 and an image acquisition device 16. The image acquisition device 16 has an image sensor, a steering pan-tilt head and a focusing module. The steering pan-tilt head rotates the image sensor of the image acquisition device 16 according to the rotation angle information to align with the lifted container 4, and after the focusing module adjusts the focal length of the image acquisition device 16 according to the focal length information, an image with at least a partial outer surface of the container 4 is taken. The spreader positioning device 15 is a 3D laser sensor, which can scan the spatial form around the crane, and a training set is established by collecting the spatial scanning data of the crane 2 lifting the container 4 through the spreader positioning device 15. As Figure 5 shown, the spatial scanning data obtained by the 3D laser sensor is a three-dimensional point cloud data set (G101, G102, G103, G104... are the laser points generated on the surface of the container 4; J101, J102, J103,... are the laser points generated on the surface of the spreader 21, and each laser point has a spatial coordinate in the laser coordinate system established by the 3D laser sensor). The position information of the spreader 21 is the three-dimensional point cloud set of the spreader 21, and the position information of the container 4 is the three-dimensional point cloud set of multiple surfaces of the container 4. In the training set, the spatial scanning data when the crane 2 lifts the container 4 and the spatial scanning data when the crane 2 does not lift the container 4 are manually marked. Through the training set, a prediction model for judging whether the crane 2 lifts the container 4 is obtained by using, for example, machine learning or neural network algorithms. The process of obtaining the obtained prediction model can use the prior art and will not be elaborated here. The spatial scanning data at least includes the position information of the spreader 21 and the position information of the container 4.

[0137] During on-site use, the spreader positioning device 15 is used to detect the spatial scanning data of the current state of the crane 2. When it is judged by the prediction model that the crane 2 has lifted the container 4, according to the position information of the container 4 in the spatial scanning data at this time, the rotation angle information and the focal length information of a target surface of the container 4 and the container recognition component 1 are obtained. Refer to Figure 6 , when the square surface at one end of the container 4 ( Figure 6The angle between the square surface (including the four points F1, F2, F3, and F4) and the container identification component 1 is within the first preset threshold. The first preset threshold can be 45°. Then, the spatial coordinates of the midpoint (F9) of the square surface are used as the target coordinates. At this time, the square surface is directly facing the container identification component 1, and the captured image is easier to identify the identification code. According to the target coordinates, the rotation angle information, and the focal length information of the container identification component 1. The steering gimbal of the image acquisition device 16 rotates the image sensor of the image acquisition device 16 according to the rotation angle information to align with the container 4. After the focusing module adjusts the focal length of the image acquisition device 16 according to the focal length information, an image with at least a partial outer surface of the container 4 is captured.

[0138] Or, referring to Figure 7 , when the angle between a rectangular surface of the container 4 ( Figure 7 the rectangular surface including the four points F1, F4, F5, and F8) and the container identification component 1 is within the second preset threshold. The second preset threshold can be 45°. Then, the spatial coordinates of the midpoint (F10) of the preset area (preset according to the position of the identification code on the side of the container) in the rectangular surface are used as the target coordinates. At this time, the rectangular surface is directly facing the container identification component 1, and the captured image is easier to identify the identification code. Then, the focal length information and the rotation angle information are obtained according to the target coordinates, and the container image is captured, so as to obtain the clearest container image, greatly improving the accuracy of subsequent container identification. It also effectively prevents the interference of other containers 3 stacked on the ground to the container identification. According to the target coordinates (F10), the rotation angle information, and the focal length information of the container identification component 1. The steering gimbal of the image acquisition device 16 rotates the image sensor of the image acquisition device 16 according to the rotation angle information to align with the container 4. After the focusing module adjusts the focal length of the image acquisition device 16 according to the focal length information, an image with at least a partial outer surface of the container 4 is captured.

[0139] Finally, perform text and image recognition on the captured image to obtain the identification code on the surface of the container 4 as "ABCD1234". According to the identification code "ABCD1234" of the container 4, obtain the preset target placement position of the container 4 (for example: cabinet position No. 21). The crane 2 hoists the container 4 to the target placement position.

[0140] The embodiment of the present invention also provides a container identification device based on spatial scanning, including a processor. A memory, in which executable instructions of the processor are stored. Wherein, the processor is configured to execute the steps of the container identification method based on spatial scanning by executing the executable instructions.

[0141] As described above, the container identification device based on spatial scanning of the present invention can realize follow-up tracking and identification of the lifted container by fusing the spreader positioning sensor to guide the follow-up image sensor, thereby realizing intelligent tallying of container operations.

[0142] Those skilled in the art can understand that various aspects of the present invention can be implemented as a system, method, or program product. Therefore, various aspects of the present invention can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which can be collectively referred to as "circuit", "module", or "platform" here.

[0143] Figure 8 It is a schematic structural diagram of the container identification device based on spatial scanning of the present invention. The following will be described with reference to Figure 8 the electronic device 600 according to this embodiment of the present invention. Figure 8 The displayed electronic device 600 is only an example and should not impose any restrictions on the functions and usage scope of the embodiments of the present invention.

[0144] As Figure 8 shown, the electronic device 600 is presented in the form of a general-purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0145] Among them, the storage unit stores program code, and the program code can be executed by the processing unit 610, so that the processing unit 610 executes the steps according to various exemplary embodiments of the present invention described in the above-mentioned electronic prescription circulation processing method part of this specification. For example, the processing unit 610 can execute the steps as Figure 1 shown.

[0146] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 6201 and / or a cache storage unit 6202, and may further include a read-only storage unit (ROM) 6203.

[0147] The storage unit 620 may further include a program / utility 6204 having a set (at least one) of program modules 6205. Such program modules 6205 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. The implementation of a network environment may be included in each or some combination of these examples.

[0148] The bus 630 can represent one or more of several types of bus structures, including a memory unit bus or a memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of the various bus structures.

[0149] The electronic device 600 can also communicate with one or more external devices 700 (such as a keyboard, a pointing device, a Bluetooth device, etc.), can also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or can communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (such as a router, a modem, etc.). Such communication can be carried out through the input / output (I / O) interface 650. Moreover, the electronic device 600 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the network adapter 660. The network adapter 660 can communicate with other modules of the electronic device 600 through the bus 630. It should be understood that although not shown in the figure, other hardware and / or software modules can be used in combination with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, etc.

[0150] An embodiment of the present invention also provides a computer-readable storage medium for storing a program, and the steps of a container recognition method based on spatial scanning are implemented when the program is executed. In some possible implementation manners, various aspects of the present invention can also be implemented in the form of a program product, which includes program code. When the program product runs on a terminal device, the program code is used to cause the terminal device to execute the steps according to various exemplary embodiments of the present invention described in the above-mentioned electronic prescription circulation processing method part of this specification.

[0151] As shown above, when the program of the computer-readable storage medium of this embodiment is executed, it can realize follow-up tracking recognition of the container being lifted by fusing the spreader positioning sensor to guide the follow-up image sensor, so as to realize intelligent tallying of container operations.

[0152] Figure 9 is a schematic structural diagram of the computer-readable storage medium of the present invention. Refer to Figure 9 As shown, a program product 800 for implementing the above method according to an embodiment of the present invention is described. It can use a portable compact disc read-only memory (CD-ROM) and includes program code, and can run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited to this. In this document, the readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, device, or device.

[0153] The program product may employ any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0154] The computer-readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, in which the readable program code is carried. Such a propagated data signal may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The readable storage medium may also be any readable medium other than the readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0155] The program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and also including conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, executed as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0156] In summary, the container identification method, system, device, and storage medium based on spatial scanning of the present invention can achieve follow-up tracking and identification of the lifted container by fusing the spreader position sensor to guide the follow-up image sensor, thereby realizing intelligent tallying of container operations.

[0157] The above content is a further detailed description of the present invention in combination with specific preferred embodiments. It cannot be determined that the specific implementation of the present invention is only limited to these descriptions. For those of ordinary skill in the technical field to which the present invention pertains, without departing from the concept of the present invention, several simple deductions or substitutions can still be made, and all should be regarded as belonging to the protection scope of the present invention.

Claims

1. A container recognition method based on spatial scanning, characterized in that Adopt at least one container identification component integrated with a spreader positioning device and an image acquisition device, including the following steps: Collect spatial scan data of the crane hoisting a container through the spreader positioning device to establish a training set, and obtain a prediction model for judging whether the crane hoists a container. The spatial scan data at least includes the position information of the spreader and the position information of the container. The spreader positioning device is a point cloud sensor, the spatial scan data is a three-dimensional point cloud data set, the position information of the spreader is a three-dimensional point cloud set on the surface of the spreader, and the position information of the container is a three-dimensional point cloud set on multiple surfaces of the container; Use the spreader positioning device to detect the spatial scan data of the current state of the crane; When it is judged by the prediction model that the crane has hoisted a container, according to the position information of the container in the spatial scan data at this time, obtain the rotation angle information and focal length information between a target surface of the container and the container identification component. Generate the spatial position information of multiple surfaces of the container according to the three-dimensional point cloud set on the surface of the container. When the angle between the square surface at one end of the container and the container identification component is within a first preset threshold, the first preset threshold is 45°, then use the spatial coordinates of the midpoint of the square surface as the target coordinates; or, when the angle between a rectangular surface of the container and the container identification component is within a second preset threshold, the second preset threshold is 45°, then use the spatial coordinates of the midpoint of a preset area in the rectangular surface as the target coordinates; Obtain the focal length information and rotation angle information according to the target coordinates, obtain the focal length information according to the distance between the spatial coordinates of the center point of the target surface of the container and the spatial coordinates of the container identification component, and obtain the rotation angle information according to the direction between the spatial coordinates of the center point of the target surface of the container and the spatial coordinates of the container identification component; The image acquisition device rotates according to the rotation angle information and takes a picture according to the focal length information; Perform graphic and text recognition on the taken picture to obtain the identification code on the surface of the container.

2. The container identification method based on spatial scanning according to claim 1, characterized in that, The image acquisition device has an image sensor, a steering pan-tilt head and a focusing module. The steering pan-tilt head rotates the image sensor of the image acquisition device to align with the container according to the rotation angle information. After the focusing module adjusts the focal length of the image acquisition device according to the focal length information, take a picture of an image with at least a partial outer surface of the container.

3. The container recognition method based on spatial scanning according to claim 1, wherein, The point cloud sensor is a laser sensor. The laser sensor performs circumferential scanning based on the crane, generates laser points on the surfaces of the spreader and the container, and uses the laser sensor as the coordinate origin to obtain the spatial coordinates of the laser points in the laser coordinate system of the laser sensor. Establish a training set according to the set of laser point spatial coordinates in the first state of the crane hoisting the container and the second state of the crane not hoisting the container.

4. The container identification method based on spatial scanning according to claim 1, wherein The spreader positioning device is one or a combination of a 2D laser sensor, a 3D laser sensor, an infrared sensor, a millimeter wave radar, a monocular or binocular vision sensor.

5. The container identification method based on spatial scanning according to claim 1, wherein The prediction model determines that the crane has lifted a container based on the simultaneous satisfaction of the conditions that the container is directly below the crane and the container moves synchronously with the crane in the horizontal direction.

6. The container identification method based on spatial scanning according to claim 1, characterized in that The container identification component is fixed on the surface of the crane.

7. The container recognition method based on spatial scanning according to claim 1, characterized in that, The crane is a gantry crane, and two container identification components are arranged on the gantry crane, one is arranged below the cockpit, and the other is arranged at the land side leg of the gantry crane.

8. The container identification method based on spatial scanning according to claim 1, characterized in that, Based on the identification code of the container, the preset target placement position of the container is obtained, and the crane hoists the container to the target placement position.

9. A container identification system based on spatial scanning, characterized in that, Used to implement the container identification method based on spatial scanning as described in claim 1, including: A container identification component, the container identification component includes a spreader positioning device and an image acquisition device, the spreader positioning device is one or a combination of a 2D laser sensor, a 3D laser sensor, an infrared sensor, a millimeter wave radar, a monocular or binocular vision sensor, the image acquisition device has an image sensor, a steering pan-tilt head and a focusing module, the steering pan-tilt head and the focusing module are respectively connected to the container identification component, the spreader positioning device is a point cloud sensor, the spatial scanning data is a three-dimensional point cloud data set, the position information of the spreader is a three-dimensional point cloud set on the surface of the spreader, the position information of the container is a three-dimensional point cloud set on multiple surfaces of the container, the spatial position information of multiple surfaces of the container is generated according to the three-dimensional point cloud set on the surface of the container, when the angle between the square surface at one end of the container and the container identification component is within the first preset threshold, the spatial coordinates of the midpoint of the square surface are used as the target coordinates; or, when the angle between a rectangular surface of the container and the container identification component is within the second preset threshold, the spatial coordinates of the midpoint of a preset area in the rectangular surface are used as the target coordinates; the focal length information and the rotation angle information are obtained according to the target coordinates, the focal length information is obtained according to the distance between the spatial coordinates of the center point of the target surface of the container and the spatial coordinates of the container identification component, the rotation angle information is obtained according to the direction between the spatial coordinates of the center point of the target surface of the container and the spatial coordinates of the container identification component, the first preset threshold is 45°, and the second preset threshold is 45°.

10. The container identification system based on spatial scanning according to claim 9, characterized in that, The container identification component is fixed on the surface of the crane, and the container identification component integrates a spreader positioning device and an image acquisition device for detecting the position of the container.

11. The container identification system based on spatial scanning according to claim 10, characterized in that The container identification component includes: A first bracket, one end of the first bracket is fixed on the surface of the crane; A duckbill bracket, arranged on the upper surface of the other end of the first bracket; A second bracket, the lower surface of the second bracket is connected to the duckbill bracket; A spreader positioning device, arranged on the upper surface of the second bracket; and An image acquisition device, arranged on the lower surface of the other end of the first bracket, and the steering pan-tilt head and the focusing module of the image acquisition device are connected to the spreader positioning device through data lines.

12. The container identification system based on spatial scanning according to claim 9, characterized in that The crane is a gantry crane, and the two container recognition components are arranged on the gantry crane, one below the cockpit and the other at the land side leg of the gantry crane.

13. A container identification device based on spatial scanning, characterized in that, Comprising: A processor; A memory in which executable instructions of the processor are stored; Wherein, the processor is configured to execute the steps of the container recognition method based on space scanning according to any one of claims 1 to 8 by executing the executable instructions.

14. A computer-readable storage medium for storing a program, characterized in that, When the program is executed, the steps of the container recognition method based on space scanning according to any one of claims 1 to 8 are implemented.

Images