Goods loading container placement control method and device, controller and storage medium

Abstract

The invention relates to the technical field of article transportation, and discloses a control method and device for placement of a cargo loading container, a controller and a storage medium, a lifting appliance is provided with an image acquisition device, and the method comprises the steps that when the lifting appliance grabs the current cargo loading container to be lowered, a top view containing the current cargo loading container and an operation machine is obtained; detecting whether a pre-arranged limiting part of the working machine exists in the top view or not, and judging whether potential safety hazards exist in the lowering process or not according to a detection result; and the lowering process of the current cargo loading container is controlled according to the judgment result. Whether a limiting component exists below the cargo loading container or whether potential safety hazards exist or not can be detected in advance, then lowering is forbidden when potential safety hazards exist, the situation that the cargo loading container is penetrated or collided by the limiting component to deform during lowering is avoided, meanwhile, the safety of the cargo loading container, cargoes and operators is guaranteed, and the working efficiency is improved. And the occurrence rate of safety accidents caused by abnormal movement of the container truck or inaccurate container placement and the like is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of article transportation, and in particular to a control method and device for placing a cargo loading container, a controller and a storage medium. BACKGROUND

[0002] In the process of placing a container on a truck (container truck), the usual steps include: 1. The spreader grips the container and moves downward; 2. The container is placed on the truck plate; 3. The spreader is unlocked; and 4. The spreader moves upward.

[0003] The prior art detects the angle and position of the container placing process to ensure the accuracy of the container placing. However, in actual scenarios, the truck has front and rear bumpers. If an unexpected situation occurs during the container placing process, such as abnormal movement of the truck or inaccurate container placing, the container may be penetrated by the bumper or deformed by collision, which may damage the container and the cargo, or even cause personal safety accidents. SUMMARY

[0004] The present application provides a control method and device for placing a cargo loading container, a controller and a storage medium to solve the problem of safety hazards caused by the limiting component when placing the cargo loading container.

[0005] In a first aspect, the present application provides a control method for placing a cargo loading container. The spreader is provided with an image acquisition device. The method comprises: when the spreader grips the current cargo loading container and lowers it, obtaining an overhead view containing the current cargo loading container and the working machine; detecting whether there is a limiting component arranged in advance by the working machine in the overhead view, and judging whether there is a safety hazard in the lowering process according to the detection result; and controlling the lowering process of the current cargo loading container according to the judgment result.

[0006] Advantages: The control method for placing a cargo loading container provided by the present application can detect whether there is a limiting component and whether there is a safety hazard in the lowering process based on the overhead view containing the current cargo loading container and the working machine when the spreader grips the current cargo loading container and lowers it, and control the lowering process based on the judgment result of the safety hazard. The present application can detect whether there is a limiting component below the cargo loading container and whether there is a safety hazard by deploying an image acquisition device on the spreader and obtaining an overhead view, and further prohibits lowering when there is a safety hazard, thereby avoiding the cargo loading container from being penetrated by the limiting component or deformed by collision when lowering, and ensuring the safety of the cargo loading container, the cargo and the working personnel, and reducing the incidence of safety accidents caused by abnormal movement of the truck or inaccurate container placing.

[0007] In one optional implementation, detecting whether there are pre-arranged limiting components of the operating machinery in the top view, and determining whether there are safety hazards in the lowering process based on the detection results, includes: detecting whether there are limiting components, the bottom plate of the operating machinery, or the pre-set cargo loading container on each side of the current cargo loading container based on the top view; if there are limiting components on at least one side, then for any first side with limiting components, determining the actual distance between the first side and the limiting components based on the top view; comparing the actual distance with a safety threshold, if the actual distance is greater than the safety threshold, then determining that there are no safety hazards on the first side, otherwise determining that there are safety hazards on the first side; if there are no safety hazards on each of the first sides, and each second side without limiting components has a bottom plate or the pre-set cargo loading container, then determining that there are no safety hazards in the lowering process.

[0008] Beneficial effects: This invention, by detecting the presence of a limiting component on the side of the cargo loading container below, further obtains the actual distance between the cargo loading container and the limiting component. This allows for the determination of whether the distance between the cargo loading container and the limiting component is too close, thereby identifying a safety hazard when the distance is too close and preventing damage to the cargo loading container from the limiting component. Furthermore, in the absence of a limiting component, it determines whether the load-bearing capacity is valid, enabling accurate identification of safety hazards under multiple operating conditions. This covers complex operating scenarios such as single-sided and double-sided limiting components, improving the rigor of the judgment logic.

[0009] In one optional implementation, for any first side with a limiting component, determining the actual distance between the first side and the limiting component based on a top view includes: for any first side with a limiting component, using a segmentation algorithm to determine a first mask of the current cargo loading container and a second mask of the limiting component in the top view; determining a first boundary pixel of the first mask and a second boundary pixel of the second mask based on the relative position of the current cargo loading container and the limiting component; determining the minimum pixel distance between the first mask and the second mask based on the first boundary pixel and the second boundary pixel; and determining the actual distance between the first side and the limiting component based on the current height of the spreader, the pixel focal length of the image acquisition device, and the minimum pixel distance.

[0010] Beneficial effects: By determining the actual distance based on pixel distance, this invention can accurately obtain the true horizontal distance between the cargo loading container and the limiting component, providing reliable data support for the judgment of safety hazards, ensuring the accuracy of the determination of safety thresholds, and avoiding safety risks caused by misjudgment of distance.

[0011] In one optional implementation, the method for determining whether there is a safety hazard during the lowering process based on the top view further includes: if there are no limiting components on each side and each side has a base plate or a pre-set cargo loading container, then it is determined that there is no safety hazard during the lowering process; if there are no limiting components, base plates, or pre-set cargo loading containers on all sides, then it is determined that there is a safety hazard during the lowering process.

[0012] Beneficial Effects: This invention clarifies safety assessment standards for scenarios involving unrestricted components, specifically those with floor plates on all sides or pre-set cargo loading containers. This adapts to the operational conditions of conventional trucks without side-restricted components. Furthermore, by verifying the integrity of the load-bearing structure, it avoids the risk of tilting or falling of cargo loading containers due to lack of load on one side. This ensures safety assessments cover all scenarios, including those with and without side-restricted components, guaranteeing the safety of container placement under different truck structures and operational requirements. In addition, by identifying and assessing potential risks associated with unrestricted components, floor plates, and pre-set cargo loading containers on all sides, this invention proactively mitigates the extreme safety risks of cargo loading containers being lowered without any load support. It quickly identifies operational anomalies such as damaged truck floor plates and abnormal truck parking, and promptly prohibits lowering to prevent chain reactions such as cargo loading container deformation, cargo damage, or spreader imbalance. This further enhances the coverage of safety hazards and ensures the compliance and safety of operational procedures.

[0013] In one optional implementation, detecting whether there are limiting components, the bottom plate of the operating machinery, or a preset cargo loading container on each side of the current cargo loading container based on the top view includes: identifying preset targets in the top view based on a pre-built image detection model, determining the position information of the preset targets, wherein the preset targets include at least limiting components, the bottom plate, the current cargo loading container, and the preset cargo loading container; and determining whether there are limiting components, the bottom plate, or the preset cargo loading container on each side of the current cargo loading container based on the position information.

[0014] Beneficial effects: This invention achieves accurate identification and location of core targets through a pre-built image detection model, ensuring the efficiency and accuracy of target identification, effectively eliminating interference from irrelevant components such as the vehicle front and battery box, focusing on core targets for safety determination, and providing high-quality data input for subsequent distance calculation and working condition determination, ensuring the stability and reliability of the entire technical solution.

[0015] In one optional implementation, the process of lowering the current cargo loading container is controlled based on the judgment result, including: if there is no safety hazard, the lowering continues until the current cargo loading container is placed; if there is a safety hazard, the lowering is stopped and an alarm message is generated.

[0016] Beneficial effects: By controlling the decentralization process, this invention enables timely response and closed-loop handling of safety hazards, improves the controllability of the work process, ensures the continuity of normal operations, and can quickly cut off dangerous processes when risks occur. Alarm information is used to remind operators to check for abnormalities and prevent accidents from escalating.

[0017] In some alternative implementations, before obtaining a top view including the current cargo loading container and the operating machinery, the method further includes: obtaining the current height of the spreader and determining whether the current height is higher than a preset threshold.

[0018] Beneficial effects: By detecting the current height of the lifting device, this invention can ensure that the collected top view contains key information, thereby determining whether there are any safety hazards during the lowering process and avoiding limited viewing angles when the lowering height is low.

[0019] Secondly, the present invention provides a control device for placing a cargo loading container. The device includes: an image acquisition module for acquiring a top view including the current cargo loading container and the operating machinery when the spreader grips and lowers the current cargo loading container; a hazard identification module for detecting whether there are pre-arranged limiting components of the operating machinery in the top view, and judging whether there are safety hazards in the lowering process based on the detection results; and a lowering control module for controlling the lowering process of the current cargo loading container based on the judgment results.

[0020] Thirdly, the present invention provides a working machine, comprising a controller, a lifting device, and a vehicle body. The lifting device is equipped with an image acquisition device, and both the lifting device and the image acquisition device are connected to the controller. The vehicle body has a base plate. The controller includes a memory and a processor, which are interconnected. The memory stores computer instructions, and the processor executes the computer instructions to perform the control method for placing cargo loading containers as described in the first aspect or any corresponding embodiment.

[0021] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the control method for placing cargo loading containers according to the first aspect or any corresponding embodiment thereof.

[0022] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to execute the control method for placing cargo loading containers according to the first aspect or any corresponding embodiment described above. Attached Figure Description

[0023] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of the structure of the operating machinery in an application scenario according to an embodiment of the present invention; Figure 2 This is a schematic flowchart of a first method for controlling the placement of cargo loading containers according to an embodiment of the present invention; Figure 3 This is a second flowchart illustrating the control method for placing cargo loading containers according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the first working condition of the control method for placing cargo loading containers according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the second working condition of the control method for placing cargo loading containers according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the third working condition of the control method for placing cargo loading containers according to an embodiment of the present invention; Figure 7 This is a schematic diagram of the fourth working condition of the control method for placing cargo loading containers according to an embodiment of the present invention; Figure 8 This is a schematic diagram of the third process of the control method for placing cargo loading containers according to an embodiment of the present invention; Figure 9 This is a schematic diagram of the overall process of the control method for placing cargo loading containers according to an embodiment of the present invention; Figure 10 This is a structural block diagram of a control device for placing cargo loading containers according to an embodiment of the present invention; Figure 11 This is a schematic diagram of the hardware structure of the controller according to an embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0027] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0028] As an optional application scenario of this invention, such as Figure 1 As shown, the operating machinery 100 includes: a controller 101, a lifting device 102, and a vehicle body 103. The lifting device 102 is equipped with an image acquisition device 104. Both the lifting device 102 and the image acquisition device 104 are connected to the controller 101. The vehicle body 103 has a base plate 1031. The lifting device 102 has four corners, and through the locking mechanisms at the four corners, it precisely engages with the four corner locking holes of the cargo loading container 105 to be placed, completing a stable grip and ensuring that the cargo loading container is in a horizontal position, preventing swaying during the descent.

[0029] In this embodiment of the invention, the operating machinery includes, but is not limited to, container trucks, loading platforms, etc. Taking a container truck as an example, the cargo container 105 is a container to illustrate this embodiment of the invention, and the limiting component is a baffle. The image acquisition device 104 is a camera, and in this embodiment of the invention, one camera is deployed at each of the four corners of the spreader 102. The four corner cameras capture real-time top-down views from different angles, covering the four corners and surrounding areas of the current container, including the vehicle body, platform, possible baffles, and pre-set containers already on the truck, forming multi-view image data and continuously transmitting it to the controller 101. The controller 101 then controls the container placement process based on the collected multi-view image data. This embodiment of the invention provides a control method for the placement of cargo loading containers, which detects whether the image data contains baffles to determine whether there are any safety hazards.

[0030] According to an embodiment of the present invention, a control method for placing cargo loading containers is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0031] This embodiment provides a control method for placing cargo loading containers, which can be used in the aforementioned operating machinery, such as container trucks.Figure 2 This is a flowchart of a control method for placing cargo loading containers according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps: Step S201: When the spreader grabs and lowers the current cargo loading container, obtain a top view including the current cargo loading container and the operating machinery.

[0032] Specifically, in this embodiment of the invention, a container truck is used as the control unit. Once the four corners of the spreader on the container truck precisely engage with the four corner locking holes of the container to be placed via the locking mechanism, achieving a secure grip, the container truck's controller receives the container release command. Based on the pre-set placement area on the truck bed, it plans the downward path of the spreader and drives the spreader, carrying the container, upwards towards the truck bed, completing the initial alignment. The spreader, carrying the container, moves vertically downwards towards the truck bed from a certain height, for example, 7 meters. During the descent, the cameras deployed at the four corners of the spreader are activated simultaneously.

[0033] In some alternative implementations, cameras at the four corners capture real-time top-down views from different angles, covering the four corners and surrounding areas of the current container, forming a multi-view top-down view. This view includes at least the current container and the container truck, but does not fully represent the entire container or the container truck. In practice, each camera focuses on the corresponding corner and surrounding area of ​​the current container; for example, the upper left camera covers the upper left corner, left side, and front perimeter of the container, while the lower right camera covers the lower right corner, right side, and rear perimeter. Therefore, the multi-view top-down view includes the core areas at the four corners of the current container and key surrounding structures, such as baffles, partial views of the truck bed, and pre-defined container views, thus satisfying inspection requirements while reducing the amount of data processed in image processing.

[0034] Step S202: Check whether there are pre-arranged limiting components of the operating machinery in the top view, and determine whether there are any safety hazards in the lowering process based on the detection results.

[0035] Specifically, in this embodiment of the invention, front and rear baffles are installed on the container truck to secure the container, protect key components, and improve transportation safety. The front baffle prevents the container from sliding forward during start-up, braking, or sharp turns, directly abutting the front corner fittings of the container and working with the container truck's twist lock to secure the front end, preventing the container from colliding with the tractor unit. The rear baffle prevents the container from sliding backward during transport (especially during braking and deceleration), locking the rear corner fittings of the container and, together with the front baffle and twist lock, forming a two-way front and rear limit for the container, ensuring that the container remains fixed in the designated position on the trailer frame.

[0036] However, if baffles are installed, when the spreader lowers the container onto the truck bed, any misalignment can cause the container to come into contact with and be squeezed by the pre-installed front and rear baffles on the truck. This can lead to container penetration and deformation, resulting in damage to the cargo inside, and in extreme cases, causing the container to fall and endangering the safety of the workers. Therefore, baffles represent a safety hazard to some extent. If baffles are currently installed, they will be visible in the top view when the container is lowered close to them. If the container is far from the baffles, they will not be visible in the top view. This embodiment of the invention detects the presence of pre-installed baffles on the container truck in the top view and determines whether there is a safety hazard during the lowering process based on the detection results.

[0037] Step S203: Control the lowering process of the current cargo loading container based on the judgment result.

[0038] Specifically, in this embodiment of the invention, if a baffle is present in the top view, it indicates a potential safety hazard during the container lowering process. Therefore, lowering must be stopped, and the lowering position adjusted to safely place the container in the correct location. If there is no safety hazard, lowering continues until the container is placed in the correct position.

[0039] The present invention provides a method for controlling the placement of cargo loading containers. When the spreader grips and lowers the cargo loading container, it detects the presence of limiting components based on a top-down view including the container and the operating machinery. The method then determines whether there are any safety hazards during the lowering process based on the detection results and controls the process accordingly. By deploying an image acquisition device on the spreader to obtain a top-down view, the present invention can detect whether there are limiting components or safety hazards below the cargo loading container. If safety hazards are found, lowering is prohibited, preventing the container from being penetrated or deformed by limiting components during lowering. This ensures the safety of the container, cargo, and personnel, and reduces the incidence of accidents caused by truck movement or inaccurate container placement.

[0040] This embodiment provides a control method for placing cargo loading containers, which can be used in the aforementioned operating machinery, such as container trucks. Figure 3 This is a flowchart of a control method for placing cargo loading containers according to an embodiment of the present invention, such as... Figure 3 As shown, the process includes the following steps: Step S301: When the spreader grips and lowers the currently loaded cargo container, obtain a top view including the currently loaded cargo container and the operating machinery. For details, please refer to [link to details]. Figure 2 Step S201 of the illustrated embodiment will not be described again here.

[0041] Step S302: Check whether there are pre-arranged limiting components of the operating machinery in the top view, and determine whether there are any safety hazards in the lowering process based on the detection results.

[0042] Specifically, step S302 includes: Step S3021: Based on the top view, detect whether there are limiting parts, the bottom plate of the operating machinery, or the preset cargo loading container on each side of the current cargo loading container.

[0043] Specifically, in this embodiment of the invention, during the container unloading process, the baffles deployed on the container truck are typically located on the front and rear sides of the vehicle. If the unloaded container is large, both the front and rear sides of the container will be close to the baffles. If the unloaded container is relatively small, it may be close to one baffle, or neither side may be close to the baffles (placed in the middle of the container truck). Therefore, the presence of baffles on each side of the current container is detected based on a top-down view, with a primary focus on the front and rear sides of the container, but not limited to this.

[0044] Meanwhile, in order to ensure the safety of the container, that is, to ensure that the container can be safely placed on the truck bed, when judging whether there is a safety hazard, we simultaneously check whether there is a truck bed or a pre-placed container on each side of the container based on the top view, and focus on the other sides without baffles.

[0045] In some optional implementations, step S3021 above includes: Step a1: Based on the pre-built image detection model, perform preset target recognition on the top view to determine the position information of the preset target. The preset target includes at least the limiting component, the bottom plate, the current cargo loading container, and the preset cargo loading container.

[0046] Step a2: Determine whether there are limiting components, bottom plates, or preset cargo loading containers on each side of the current cargo loading container based on the location information.

[0047] Specifically, in this embodiment of the invention, an image detection model is pre-constructed based on deep learning algorithms, such as YOLO and Faster R-CNN. This image detection model has been trained with a large number of top-view samples of port operation scenarios, possessing the ability to accurately identify key targets in complex backgrounds. The pre-defined targets clearly include at least baffles, truck platforms, current containers, and pre-defined containers (i.e., containers already loaded on the truck). It is also compatible with identifying auxiliary targets such as the truck cab, battery boxes, and connection points for subsequent interference elimination. In practical applications, multi-view top-view images captured by the four corner cameras of the spreader are simultaneously input into the image detection model. The model analyzes the images frame by frame, distinguishing and identifying various pre-defined targets based on their visual features (such as contours, textures, and grayscale differences), thus completing the initial differentiation between targets and background, and between different targets.

[0048] The image detection model outputs precise location information of various preset targets in the image, which is presented in the form of pixel coordinate ranges. Based on the current container's location information, the various sides of the current container are determined. Then, by combining the location information of baffles, platform panels, and preset containers, it is determined whether baffles, platform panels, or preset containers exist on each side.

[0049] Step S3022: If there is a limiting component on at least one side, then for any first side with a limiting component, determine the actual distance between the first side and the limiting component based on the top view.

[0050] Specifically, in this embodiment of the invention, if a baffle exists on at least one of the front and rear sides of the container, the side with the baffle is designated as the first side. The presence of a baffle only indicates a potential safety hazard; to further determine whether the baffle poses a safety hazard to the container, the actual distance between the first side and the baffle needs to be considered.

[0051] In some optional implementations, step S3022 above includes: Step b1: For any first side with a limiting component, use a segmentation algorithm to determine the first mask of the current cargo loading container and the second mask of the limiting component in the top view.

[0052] Step b2: Based on the relative position of the current cargo loading container and the limiting component, determine the first boundary pixel of the first mask and the second boundary pixel of the second mask.

[0053] Step b3: Determine the minimum pixel distance between the first mask and the second mask based on the first boundary pixel and the second boundary pixel.

[0054] Step b4: Determine the actual distance between the first side and the limiting component based on the current height of the lifting device, the pixel focal length of the image acquisition device, and the minimum pixel distance.

[0055] Specifically, in this embodiment of the invention, the top view obtained can show the distance between the first side and the baffle, but it can only visually show the pixel-level spacing between the first side and the baffle, that is, the pixel interval between the edges of the two targets in the image, which cannot be directly used as a basis for safety determination. Therefore, this embodiment of the invention converts the pixel spacing into the actual distance in physical space, and uses this as the basis for safety determination.

[0056] This invention identifies key viewpoint images containing baffles in multi-angle top-view images and inputs these images into a pre-built segmentation model. The segmentation model has been trained on a large number of port operation scene images, including top-view images with different lighting conditions, different truck models, and different baffle styles. It can accurately identify the visual features of containers and baffles, such as the rectangular outline and metallic grayscale texture of containers, and the striped structure and grayscale difference between the baffles and the background. Furthermore, the segmentation model performs pixel-by-pixel classification on the current key viewpoint image, obtaining two independent binary masks, including the first mask for the current container. and the second mask of the baffle This allows for the segmentation of the current container and the baffle. The mask size is the same as the input image, both being H×W pixel matrices. In the pixel matrix corresponding to the first mask, This indicates that the pixel belongs to the pixel area of ​​the current container. This represents the non-container area (background area); in the pixel matrix corresponding to the second mask... This indicates that the pixel belongs to the pixel area of ​​the baffle. This represents the non-baffle area (background area). This embodiment of the invention uses a double mask to achieve precise pixel-level isolation between the current container and the corresponding baffle.

[0057] Furthermore, in this embodiment of the invention, the edge pixels that best reflect the actual spacing are selected from the target area of ​​the mask, namely the edges of the first side of the container adjacent to the baffle, rather than all edges of the entire target. Therefore, the relative position of the container and the baffle is initially determined by the obtained double mask. Observation is performed from the side of the container truck, such as... Figure 4 As shown, if the first side is the left side of the container, then the baffle is located to the left of the left side mask of the container; as Figure 5 As shown, if the first side is the right side of the container, then the baffle is located to the right of the container's right-side mask; as Figure 6As shown, the first side includes the left and right sides of the container (relative to the front and rear sides of the container truck), so the baffle is located on the left side of the left side mask of the container and on the right side of the right side mask of the container, respectively. Then the first boundary pixel of the first mask and the second boundary pixel of the second mask are determined. Here, the boundary pixel refers to the intersection point of the target pixel (1) and the background pixel (0) in the mask, that is, the edge contour pixel of the target in the image.

[0058] For example, extract the first boundary pixel on the current container side: based on the relative position, extract only the first mask. For the boundary pixels on the side facing the baffle, if there is a baffle on the left, then extract the left edge pixels of the container mask to form the first boundary pixel set. :

[0059] in, Represents pixels The 8-neighborhood, that is, for any pixel in the mask. If at least one pixel in its 8 neighboring areas (top, bottom, left, right, and four diagonally adjacent pixels) has a value of 0, then that pixel... For boundary pixels.

[0060] Similarly, extract the second boundary pixels on the side of the baffle: only extract the boundary pixels of the second mask facing the container. If there is a baffle on the left, extract the right edge pixels of the baffle mask to form the second boundary pixel set. :

[0061] The embodiments of the present invention focus only on the edge pixels of two adjacent targets, thereby avoiding invalid calculations on non-adjacent edges, which improves calculation efficiency and ensures that the distance calculated in the subsequent calculation is the true distance between the nearest edges of the two targets, rather than the center distance of the region or the distance between non-adjacent edges.

[0062] For the first boundary pixel set any pixel in , with the second boundary pixel set any pixel in The Euclidean distance between two pixels, representing the straight-line distance at the pixel level, is calculated using the following formula:

[0063] Traversing by algorithm All pixels in Calculate the Euclidean distance for each pair of pixels in the set of all pixels. Then from all The minimum value is selected from the options; this is the minimum pixel distance.

[0064] in, In the top view, this represents the number of pixels between the nearest points of the first side edge of the container and the edge of the baffle. =5, meaning there is a 5-pixel gap between the nearest points. However This refers only to the pixel-level distance, which is affected by the camera's imaging ratio. For the same physical distance, the closer the camera is to the target, the greater the pixel distance. Therefore, this embodiment of the invention utilizes the current height of the lifting device... Pixel focal length of the image acquisition device Minimum pixel distance This is converted to the actual horizontal distance between the first side of the container and the baffle in physical space. Ultimately, this provides a directly comparable quantitative basis for safety assessment, and the conversion formula is shown below:

[0065] Furthermore, if the selected multiple viewpoints all cover the first side and the baffle, then it is necessary to calculate the values ​​for each viewpoint separately. .

[0066] In some alternative implementations, embodiments of the present invention may also use lasers to acquire point clouds of the baffle and container, analyze the distance between the container and the baffle, and then determine whether the container will be penetrated or deformed, without limitation.

[0067] Step S3023: Compare the actual distance with the safety threshold. If the actual distance is greater than the safety threshold, it is determined that there is no safety hazard on the first side; otherwise, it is determined that there is a safety hazard on the first side.

[0068] Specifically, in this embodiment of the invention, a safety threshold is preset, for example, 3cm, and the actual distance between the container and the baffle is compared with the safety threshold. If the actual distance is greater than the safety threshold, it proves that although the baffle exists, it will not damage the container, and therefore it is determined that there is no safety hazard on the first side. If the actual distance is not greater than the safety threshold, it proves that the baffle is very likely to damage the container, and therefore it is determined that there is a safety hazard on the first side.

[0069] Step S3024: If there are no safety hazards on each of the first sides, and each of the second sides without limiting components has a base plate or a pre-set cargo loading container, then it is determined that there are no safety hazards in the lowering process.

[0070] Specifically, in this embodiment of the invention, taking the left and right sides of the current container as an example, such as... Figure 4 and Figure 5As shown, the container currently only has baffles on the left or right side; therefore, it is sufficient to determine whether there are any safety hazards posed by these baffles on the left or right side. While the other side does not have a baffle, to ensure load-bearing safety, it is necessary to determine whether there is a platform or a pre-installed container. For example... Figure 6 As shown, there are baffles on both the left and right sides of the container. Therefore, it is necessary to determine whether there are any safety hazards caused by the baffles on both sides, but it is not necessary to determine whether there is a platform or a pre-installed container. In summary, if there are no safety hazards on each of the first sides (the side with the baffle) and there is a platform or a pre-installed container on each of the second sides (the side without the baffle), then it is determined that there are no safety hazards in the lowering process; if there is a safety hazard on any of the first sides (actual distance ≤ 3cm), or if there is a safety hazard on any of the second sides (no platform or pre-installed container), then it is determined that there are safety hazards in the lowering process.

[0071] In one alternative implementation, such as Figure 7 As shown, the container is currently positioned in the middle of the platform, so the left and right sides are far from the barriers. Therefore, the barriers cannot be detected in the top view. However, platforms on both sides are visible, so the lowering process is deemed safe. If there are no barriers, platforms, or pre-installed containers on any of the container's sides, the lowering position is too off-center, thus posing a safety hazard.

[0072] Step S303: Control the lowering process of the current cargo loading container based on the judgment result. See details below. Figure 2 Step S203 of the illustrated embodiment will not be described again here.

[0073] The present invention provides a method for controlling the placement of cargo loading containers. When the spreader grips and lowers the cargo loading container, it detects the presence of limiting components based on a top-down view including the container and the operating machinery. The method then determines whether there are any safety hazards during the lowering process based on the detection results and controls the process accordingly. By deploying an image acquisition device on the spreader to obtain a top-down view, the present invention can detect whether there are limiting components or safety hazards below the cargo loading container. If safety hazards are found, lowering is prohibited, preventing the container from being penetrated or deformed by limiting components during lowering. This ensures the safety of the container, cargo, and personnel, and reduces the incidence of accidents caused by truck movement or inaccurate container placement.

[0074] This embodiment provides a control method for placing cargo loading containers, which can be used in the aforementioned operating machinery, such as container trucks. Figure 8 This is a flowchart of a control method for placing cargo loading containers according to an embodiment of the present invention, such as... Figure 8 As shown, the process includes the following steps: Step S801: Obtain the current height of the spreader and determine whether the current height is higher than a preset threshold.

[0075] Specifically, in this embodiment of the invention, to ensure the safety of container lowering, the container is generally lowered from a relatively high position, for example, starting from a height of 7 meters. During the lowering process, cameras are deployed at the four corners of the spreader to capture a top-down view. If the spreader height is too low, i.e., the container is too close to the truck bed, the camera's field of view will be limited, making it impossible to capture crucial information. Therefore, as... Figure 9 As shown, the current height of the spreader is acquired in real time during the lowering process, and it is determined whether the current height is higher than a preset threshold, such as 3 meters. The preset threshold is determined based on the camera's perspective to ensure that the acquired images contain the necessary key information, such as baffles, platform panels, or placed containers.

[0076] Step S802: When the spreader grips and lowers the currently loaded cargo container, obtain a top view including the currently loaded cargo container and the operating machinery. For details, please refer to [link to relevant documentation]. Figure 3 Step S301 of the illustrated embodiment will not be described again here.

[0077] Step S803: Check the top view for any pre-installed limiting components on the working machinery, and determine whether there are any safety hazards during the lowering process based on the inspection results. For details, please refer to [link to relevant documentation]. Figure 3 Step S302 of the illustrated embodiment will not be described again here.

[0078] Step S804: Control the lowering process of the current cargo loading container based on the judgment result.

[0079] Specifically, step S804 includes: In step S8041, if there are no safety hazards, continue lowering the container until the current cargo loading container is placed.

[0080] Specifically, in this embodiment of the invention, when it is determined that there are no safety hazards on any side of the current container, that is, the actual distance on the side with the baffle... D > When the safety threshold (3cm) is reached, if a vehicle platform or a pre-set container is present on the side without the baffle, the controller issues a continuous lowering command to the spreader. During the lowering process, the image acquisition device carried by the spreader continuously acquires a top-down view. The image detection model and distance calculation module work synchronously to update the current height H of the spreader and recalculate the actual distance D in real time, ensuring that there are no new safety hazards throughout the process.

[0081] Furthermore, when the image detection model identifies that the bottom edge of the current container is completely aligned with the truck bed or the top of the preset container, and the four corner locking holes of the container are precisely aligned with the locking seats on the truck bed, the placement is determined to be complete, and the controller sends a stop lowering command to the spreader. After the spreader stops lowering, an unlocking operation is performed to release the current container. Subsequently, the controller drives the spreader to move upward away from the placement area, the image acquisition device stops working, and the placement operation is completed.

[0082] In step S8042, if there is a safety hazard, the lowering process is stopped and an alarm message is generated.

[0083] Specifically, in this embodiment of the invention, when the actual distance on any side with a baffle... D If the distance is ≤ the safety threshold (3cm), or if there is no platform / preset container load on the side without baffle, a safety hazard is determined. The controller issues an emergency stop command to the spreader. The spreader immediately brakes and maintains its current height to prevent the container from continuing to move down and causing it to collide with the baffle, penetrate and deform, or tilt and fall due to lack of load support.

[0084] At the same time, such as Figure 9 As shown, a multi-dimensional alarm mechanism is triggered simultaneously, such as on-site audible and visual alarms and system platform alarms, to ensure timely response by operators. After an alarm is triggered, operators must first check and eliminate safety hazards, such as adjusting the parking position of the truck, clearing obstructions from the baffle, and repairing damaged truck beds. After the hazards are eliminated, the image acquisition and distance calculation process is restarted. Only after it is determined that there are no hazards again can the lowering operation of the lifting equipment be resumed.

[0085] In some optional embodiments, the present invention may also, after determining that there is a safety hazard caused by the baffle and stopping the lowering, have the controller adjust the lowering position according to the actual distance so that the current container is away from the baffle, and resume the lowering operation of the spreader after the safety hazard is eliminated.

[0086] The present invention provides a method for controlling the placement of cargo loading containers. When the spreader grips and lowers the cargo loading container, it detects the presence of limiting components based on a top-down view including the container and the operating machinery. The method then determines whether there are any safety hazards during the lowering process based on the detection results and controls the process accordingly. By deploying an image acquisition device on the spreader to obtain a top-down view, the present invention can detect whether there are limiting components or safety hazards below the cargo loading container. If safety hazards are found, lowering is prohibited, preventing the container from being penetrated or deformed by limiting components during lowering. This ensures the safety of the container, cargo, and personnel, and reduces the incidence of accidents caused by truck movement or inaccurate container placement.

[0087] This embodiment also provides a control device for placing cargo loading containers, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0088] This embodiment provides a control device for placing cargo loading containers, such as... Figure 10 As shown, it includes: The image acquisition module 1001 is used to acquire a top view including the current cargo loading container and the operating machinery when the spreader grabs and lowers the current cargo loading container.

[0089] The hazard identification module 1002 is used to detect whether there are pre-arranged limiting components of the operating machinery in the top view, and to determine whether there are safety hazards in the lowering process based on the detection results.

[0090] The release control module 1003 is used to control the release process of the current cargo loading container based on the judgment result.

[0091] In some optional implementations, the hazard identification module 1002 includes: The image recognition unit is used to detect, based on a top view, whether there are limiting components, the bottom plate of the operating machinery, or a pre-set cargo loading container on each side of the current cargo loading container.

[0092] The distance calculation unit is used to determine the actual distance between the first side and the limiting component based on a top view, for any first side with a limiting component, if at least one side has a limiting component.

[0093] The distance comparison unit is used to compare the actual distance with the safety threshold. If the actual distance is greater than the safety threshold, it is determined that there is no safety hazard on the first side; otherwise, it is determined that there is a safety hazard on the first side.

[0094] The first hazard determination unit is used to determine that there is no safety hazard in the lowering process if there are no safety hazards on each of the first sides and each of the second sides without limiting components has a base plate or a pre-set cargo loading container.

[0095] In some optional implementations, the image recognition unit includes: The target recognition subunit is used to perform preset target recognition on the top view based on a pre-built image detection model, and determine the position information of the preset target. The preset target includes at least a limiting component, a base plate, the current cargo loading container, and a preset cargo loading container.

[0096] The target determination subunit is used to determine whether there are limiting components, bottom plates, or preset cargo loading containers on each side of the current cargo loading container based on the location information.

[0097] In some alternative implementations, the distance calculation unit includes: The image mask determination subunit is used to determine, using a segmentation algorithm, the first mask of the current cargo loading container and the second mask of the limiting component in the top view for any first side with a limiting component.

[0098] The boundary pixel determination subunit is used to determine the first boundary pixel of the first mask and the second boundary pixel of the second mask based on the relative position of the current cargo loading container and the limiting component.

[0099] A pixel distance determination unit is used to determine the minimum pixel distance between a first mask and a second mask based on a first boundary pixel and a second boundary pixel.

[0100] The actual distance determination unit is used to determine the actual distance between the first side and the limiting component based on the current height of the lifting device, the pixel focal length of the image acquisition device, and the minimum pixel distance.

[0101] In some optional implementations, the hazard identification module 1002 further includes: The second hazard determination unit is used to determine that there is no safety hazard in the lowering process if there are no limiting components on each side and there is a base plate or a pre-set cargo loading container on each side.

[0102] The third hazard assessment unit is used to determine that there is a safety hazard in the lowering process if there are no limiting components, bottom plates, or pre-set cargo loading containers on all sides.

[0103] In some alternative implementations, the decentralization control module 1003 includes: The first lowering control unit is used to continue lowering the cargo container until it is completely placed if there are no safety hazards.

[0104] The second control unit is used to stop the deployment and generate an alarm message if there is a safety hazard.

[0105] In some optional embodiments, the device further includes a height determination module, used to obtain the current height of the lifting device and determine whether the current height is higher than a preset threshold.

[0106] The cargo loading container placement control device provided in this embodiment of the invention can execute the cargo loading container placement control method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the above modules and units are the same as in the corresponding embodiments described above, and will not be repeated here.

[0107] Figure 11 This is a schematic diagram of the structure of a controller in a work machine provided in an embodiment of the present invention.

[0108] The following is a detailed reference. Figure 11 The diagram illustrates a structural schematic suitable for implementing a controller in an embodiment of the present invention. The controller may include a processor (e.g., a central processing unit, graphics processing unit, etc.) 1101, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1102 or a program loaded from memory 1108 into random access memory (RAM) 1103. The RAM 1103 also stores various programs and data required for controller operation. The processor 1101, ROM 1102, and RAM 1103 are interconnected via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0109] Typically, the following devices can be connected to I / O interface 1105: input devices 1106 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 1107 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; memory 1108 including, for example, magnetic tape, hard disk, etc.; and communication devices 1109. Communication device 1109 allows the controller to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 11 A controller with various devices is shown, but it should be understood that it is not required to implement or have all of the devices shown, and may alternatively implement or have more or fewer devices.

[0110] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 1109, or installed from a memory 1108, or installed from a ROM 1102. When the computer program is executed by the processor 1101, it performs the functions defined in the control method for placing cargo loading containers according to embodiments of the present invention.

[0111] Figure 11 The controller shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0112] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the control method for placing cargo loading containers shown in the above embodiments is implemented.

[0113] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0114] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A control method of placement of a cargo loading container, characterized by, The lifting appliance is provided with an image acquisition device, and the method comprises: When the lifting appliance is lowering a current cargo loading container, an overhead view containing the current cargo loading container and the working machine is acquired; It is detected whether the overhead view contains a limiting component previously arranged by the working machine, and it is determined whether there is a safety hazard in the lowering process according to the detection result; The lowering process of the current cargo loading container is controlled according to the determination result.

2. The method of claim 1, wherein, The detection of whether the overhead view contains the limiting component previously arranged by the working machine, and the determination of whether there is a safety hazard in the lowering process according to the detection result, comprises: Based on the overhead view, it is detected whether each side of the current cargo loading container contains the limiting component, the bottom plate of the working machine or a preset cargo loading container; If the limiting component exists on at least one side, for any first side containing the limiting component, the actual distance between the first side and the limiting component is determined according to the overhead view; The actual distance is compared with a safety threshold value, if the actual distance is greater than the safety threshold value, it is determined that the first side does not have a safety hazard, otherwise it is determined that the first side has a safety hazard; If each of the first sides does not have a safety hazard, and each second side without the limiting component contains the bottom plate or the preset cargo loading container, it is determined that the lowering process does not have a safety hazard.

3. The method of claim 2, wherein, The determination of the actual distance between the first side and the limiting component for any first side containing the limiting component according to the overhead view, comprises: For any first side containing the limiting component, a segmentation algorithm is used to determine a first mask of the current cargo loading container and a second mask of the limiting component in the overhead view; According to the relative position of the current cargo loading container and the limiting component, a first boundary pixel of the first mask and a second boundary pixel of the second mask are determined; The minimum pixel distance of the first mask and the second mask is determined according to the first boundary pixel and the second boundary pixel; The actual distance between the first side and the limiting component is determined according to the current height of the lifting appliance, the pixel focal length of the image acquisition device and the minimum pixel distance.

4. The method of claim 2, wherein, The determination of whether there is a safety hazard in the lowering process according to the overhead view, the method further comprises: If each side does not contain the limiting component, and each side contains the bottom plate or the preset cargo loading container, it is determined that the lowering process does not have a safety hazard; If all sides do not contain the limiting component, the bottom plate and the preset cargo loading container, it is determined that the lowering process has a safety hazard.

5. The method of claim 2, wherein, The detection of whether each side of the current cargo loading container contains the limiting component, the bottom plate of the working machine or a preset cargo loading container based on the overhead view, comprises: The overhead view is subjected to preset target recognition based on a pre-constructed image detection model, and the position information of the preset target is determined, the preset target at least including the limiting component, the bottom plate, the current cargo loading container and the preset cargo loading container; According to the position information, it is determined whether each side of the current cargo loading container has the limiting component, the bottom plate or the preset cargo loading container.

6. The method of claim 1, wherein, According to the determination result, the lowering process of the current cargo loading container is controlled, including: If there is no safety hazard, the lowering continues until the current cargo loading container is placed; If there is a safety hazard, the lowering is stopped, and an alarm information is generated.

7. The method of claim 1, wherein, Before obtaining the overhead view containing the current cargo loading container and the working machine, the method further includes: Obtaining the current height of the lifting tool, and determining whether the current height is higher than a preset threshold.

8. A control device for placement of a cargo loading container, characterized by, The lifting tool is provided with an image acquisition device, and the device includes: An image acquisition module is configured to obtain an overhead view containing the current cargo loading container and the working machine when the lifting tool holds the current cargo loading container during lowering; A hidden danger identification module is configured to detect whether the overhead view has the limiting component arranged in advance by the working machine, and determine whether there is a safety hazard in the lowering process according to the detection result; A lowering control module is configured to control the lowering process of the current cargo loading container according to the determination result.

9. A work machine characterized by, It includes: A controller, a lifting tool and a vehicle body, the lifting tool is provided with an image acquisition device, the lifting tool and the image acquisition device are connected with the controller, and the vehicle body has a bottom plate; The controller includes a memory and a processor, which are connected with each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the control method of the cargo loading container placement according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions, and the computer instructions are used to make the computer execute the control method of the cargo loading container placement according to any one of claims 1 to 7. The computer readable storage medium stores computer instructions, and the computer instructions are used to make the computer execute the control method of the cargo loading container placement according to any one of claims 1 to 7.

Images