Automatic cargo hooking and unhooking method and system based on machine vision technology

Abstract

The invention discloses an automatic cargo hooking and unhooking method and system based on a machine vision technology. The method comprises the following steps: collecting data; preprocessing data, and respectively establishing disturbance terms of each factor according to each related data of the hook and the related data of the environment; constructing a hook coefficient model and a pressure coefficient model fusing related disturbance terms; generating an instruction; setting a hook coefficient threshold value of each level, a pressure coefficient threshold value and a double-coefficient joint threshold value, judging whether an unhooking condition is met or not by adopting a double-coefficient joint decision-making mechanism, and sending an early warning signal if the unhooking condition is not met; different processing tasks are triggered according to the unhooking instruction or the early warning signal; according to the method and the system, the unhooking composite risk can be comprehensively evaluated, the dependence of traditional mechanical positioning on standard cargo types and operation experience is broken through, intelligent unhooking decision under a complex environment and multi-dimensional disturbance can be realized, the expansion rate of adaptive cargo types is high, and the unhooking safety is guaranteed; according to the unhooking method, automatic unhooking can be achieved, and the'last kilometer 'breakpoint of port intelligent construction is broken through.

Description

Technical Field

[0001] The present invention relates to the technical field of automated hoisting, and in particular to a method and system for automatically unhooking cargo based on machine vision technology. Background Art

[0002] As the automation rate of container terminals has increased to over 90%, the lifting and transportation links at the terminals have become unmanned, but the uncoupling link is still not intelligent enough and still relies on manual intervention in complex environments, becoming the "last mile" breakpoint in the construction of smart ports.

[0003] Manual unhooking is difficult, risky, and the operating environment is not safe enough. Traditional manual operation no longer meets the development needs of intelligent production and transportation. Semi-automated equipment is used to physically isolate personnel from dangerous operating areas, and manual remote control and overall judgment are performed. This solves the safety and efficiency problems of traditional manual operation to a certain extent, but there are still certain limitations. First, there are structural defects in environmental adaptability and positioning accuracy. Existing mechanical positioning systems mostly rely on rigid structures to match standard cargo types. It is difficult to maintain stable alignment in non-standard scenarios such as stacking and deformation of bagged cargo and shaking of ships. Remote operation relies too much on experience for subjective perception of the complex environment of the cargo, and cannot perform spatial alignment under dynamic changes and environmental disturbances. , such as the judgment of the influence of disturbance factors such as cargo position deviation, swing, and environmental wind force on the accuracy of hook removal, and the expansion rate of cargo types is low; secondly, there is a lack of monitoring of device status and mechanical safety. Compared with traditional manual hook removal, remote operators cannot timely understand the safety and status of the hook. It is difficult to judge compound risks by monitoring only mechanical parameters, such as the inability to grasp the factors affecting successful hook removal, such as the locking status of the hook, the health status of the hook, and the load rate; thirdly, the safety warning mechanism is simple, and the coupling degree between the manual verification link and the equipment control is low. The single-dimensional threshold judgment mechanism is difficult to capture the compound risks under complex working conditions. The accident warning is single, the response time is long, and the false trigger rate is high. The delay in emergency braking execution can easily lead to an increase in the equipment damage rate.

[0004] Therefore, there is an urgent need for a method and system for automatically unhooking cargo based on machine vision technology to solve the above problems. Summary of the Invention

[0005] The purpose of the present invention is to overcome the defects of the prior art and provide a method and system for automatically unhooking cargo based on machine vision technology to solve the problems raised in the above background technology.

[0006] In order to achieve the above object, the technical solution of the present invention is as follows:

[0007] A method for automatically unhooking cargo based on machine vision technology, comprising the following steps:

[0008] S1. Data Collection: The three-dimensional coordinates of the hook and eye are collected using visual technology. Hook-related data and environmental data are collected using multi-source perception. Hook-related data includes the hook's inclination, pressure, load, and equipment status. Environmental data includes ambient wind speed.

[0009] S2. Data preprocessing: Establish a dynamic three-dimensional coordinate system based on the three-dimensional position coordinates of the lifting ring, calculate the position deviation of the hook, and establish at least a position deviation disturbance term, a swing disturbance term, a wind resistance disturbance term, a force balance disturbance term, a load matching disturbance term, and an equipment health disturbance term based on the hook-related data and the environment-related data;

[0010] S3. Construct a coefficient model integrating relevant disturbance terms: at least integrate the position deviation disturbance term, the swing disturbance term, and the wind resistance disturbance term to establish a hook coefficient model and calculate the hook coefficient; at least integrate the force balance disturbance term, the load matching disturbance term, and the equipment health disturbance term to establish a pressure coefficient model and calculate the pressure coefficient;

[0011] S4. Generate instructions: A dual-coefficient joint decision-making mechanism is used to make decoupling decisions. Hook coefficient thresholds, pressure coefficient thresholds, and dual-coefficient joint thresholds are set at all levels. If the hook coefficient and pressure coefficient meet the decoupling conditions, a decoupling instruction is issued. If the decoupling conditions are not met, a warning signal is issued.

[0012] S5. Trigger different processing tasks according to the hook removal instruction or early warning signal.

[0013] Furthermore, it also includes the upgrading of coefficient models and iterative optimization of thresholds, setting of optimization trigger mechanisms, including active optimization, periodic optimization and event-driven optimization; active optimization fine-tunes the coefficient model based on incremental data, introduces transfer learning, and migrates the off-site coefficient models verified to be effective and anti-interference at other terminals to the local area for optimization; periodic optimization includes: setting the confidence interval of credible data according to the preset operation cycle or preset operation volume, dynamically adjusting the threshold, and mining potential correlation factors based on credible data, and introducing compound influencing indicators to upgrade the coefficient model; event-driven optimization triggers special optimization according to the number of misjudgment types or special cargo lifting.

[0014] Furthermore, the hook coefficient model is:

[0015] Hook Coefficient ;

[0016] Position deviation coefficient ;

[0017] Swing coefficient ;

[0018] Drag compensation coefficient ;

[0019] in, 、 and are the weight coefficients of the position deviation disturbance term, the swing disturbance term, and the wind resistance disturbance term, respectively, and + + =1; is the three-dimensional position deviation between the hook and the cargo ring, that is ( , , ), using the Euclidean norm The modulus length measures the three-dimensional position deviation, is the maximum allowable deviation modulus; is the hook swing angle, k is the angle suppression factor; is the real-time wind speed, is the critical wind speed.

[0020] Furthermore, the pressure coefficient model is:

[0021] Pressure coefficient ;

[0022] Force balance coefficient ;

[0023] Load matching coefficient ;

[0024] Equipment health coefficient ;

[0025] in, 、 and are the weight coefficients of the force balance disturbance term, load matching disturbance term and equipment health disturbance term, respectively, and + + =1; The real-time contact pressure of the hook, For optimal contact pressure, is the maximum allowable contact pressure, is the minimum permissible contact pressure; is the weight of the cargo, The rated load of the hook; is the actual working current of the hook electromagnetic lock, It is the nominal current of the hook electromagnetic lock.

[0026] Furthermore, when other working environment interferences other than ambient wind speed occur, the environmental adaptation weight is increased, including the following:

[0027] 1) Detect rainfall and rainfall intensity When , the weight coefficient of the position deviation disturbance term is increased to generate an adaptive weight coefficient ':

[0028] ;

[0029] 2) Haze weather is detected, and visibility V and visibility V < Increase the weight coefficient of the wind resistance disturbance term to generate an adaptive weight coefficient ':

[0030] ;

[0031] in, For minimum visibility, Normal visibility.

[0032] Furthermore, the unhooking threshold h of the hook coefficient and the unhooking threshold p of the pressure coefficient are preset. When H≤h and P≤p, the unhooking condition is met and an unhooking instruction is sent. At least three warning levels that do not meet the unhooking condition are set to generate different decision instructions. The unhooking urgency thresholds h1 and h2 of the hook coefficient, the unhooking urgency thresholds p1 and p2 of the pressure coefficient, and the dual-coefficient joint thresholds t and T are preset. The warning level is set to:

[0033] Warning level: , and H+P≤t; or, , and H+P≤t; triggering state monitoring enhancement to improve operation accuracy;

[0034] Intervention level: ;or ; or H+P ;Trigger emergency operation mode and manual confirmation process;

[0035] Emergency level: H> ; or P> ; or H+P> ; trigger emergency braking, cut power and send alarm.

[0036] Furthermore, the loss is quantified for the misjudgment types that reach a certain number of times, and the threshold is rebalanced through the ROC curve. The optimization balance process includes the following: the number of misjudgments of a certain type of dangerous event is defined as W, and the efficiency loss weight of the misjudgment is , the number of correct identification and triggering is S, the number of missed reports is L, and the accident loss weight of missed reports is , the normal number of times without false triggering is Z, and the loss function is established:

[0037] ;

[0038] Calculate the total loss at each threshold t (t), with the horizontal axis X=W / (W+S) and the vertical axis Y=L / (L+H), create an ROC curve and calculate the optimal threshold that minimizes the total loss :

[0039] .

[0040] Furthermore, for overweight special cargo, i.e. M> hour, >1, the load compensation mechanism is triggered, and the overweight cargo pressure coefficient threshold P' is used to allow short-term overload operations:

[0041] .

[0042] A cargo automatic hook removal system, wherein the hook is an automatic unlocking hook with a double lock, wherein the main lock is an electromagnetic lock and the backup lock is a mechanical lock. The automatic hook removal system is connected to the lifting control system, which includes a perception system, an intelligent analysis module, a hierarchical execution control module and a storage module; wherein,

[0043] The perception system includes a visual sensor group, an inclination sensor, a gyroscope IMU, a digital anemometer, a six-dimensional force sensor, a pressure sensor group, a current sensor and a laser scanner; the visual sensor group includes a binocular camera and a ToF depth sensor installed at the front end of the gantry crane beam or boom, which are used to scan the three-dimensional position data of the hook and the lifting ring in real time; the inclination sensor and gyroscope IMU are installed at the connection between the hook and the spreader to collect and monitor the swing angle and amplitude of the spreader; the digital anemometer is installed at the top of the boom to collect real-time wind speed data; the six-dimensional force sensor is installed at the connection between the hook and the lifting ring to collect the contact pressure data of the hook; the pressure sensor group is built into the spreader to collect cargo weight data; the current sensor is connected to the electromagnetic lock control circuit to collect the current data of the electromagnetic lock; the laser scanner is installed on the hook to capture the unlocking and locking process of the hook;

[0044] The intelligent analysis module includes a disturbance item calculation unit, a coefficient fusion calculation unit, a decision unit, and an optimization unit. The disturbance item calculation unit calculates various disturbance items based on the data collected by the perception system. The coefficient fusion calculation unit calculates the hook coefficient and pressure coefficient based on the preset or optimized weight coefficient and coefficient model of the optimization unit. The decision unit analyzes the results of the coefficient fusion calculation unit according to the preset threshold and generates corresponding control instructions. The optimization unit triggers the optimization of the weight coefficient or coefficient model according to the optimization trigger mechanism based on the real-time data, historical data, and incremental data in the storage module.

[0045] The hierarchical execution control module includes a hierarchical control end and a hierarchical execution end. The hierarchical execution end includes an electromagnetic lock, a mechanical lock, a perception system and a manual inspection and guidance system. The hierarchical control end triggers the electromagnetic lock to unlock according to the hook-removing instruction of the decision-making unit; triggers the perception system to strengthen monitoring according to the early warning level instruction, and sends a signal to improve the operation accuracy to the lifting control system; triggers the manual inspection and guidance system according to the intervention level instruction, activates the backup lock and sends an emergency operation mode signal to the lifting control system to improve the safety of abnormal operations and introduce human intervention; according to the emergency level signal to the lifting control system.

[0046] Furthermore, the storage module uses a blockchain log structure to store trusted data and adopts a consortium chain architecture, with terminal managers, equipment manufacturers, and regulatory agencies as consensus nodes. The blockchain log structure includes:

[0047] Operation fingerprint: including timestamp, spreader ID, cargo type, and environmental tag;

[0048] Raw data from the perception system: including visual positioning deviation, hook pressure value, and electromagnetic lock current waveform fragment;

[0049] Decision-making process data: including double coefficient calculation values, threshold determination results, and execution actions;

[0050] Manual correction records: include parameter adjustment contents and reasons when the operator intervenes.

[0051] Compared with the prior art, the present invention's automatic cargo hook removal method and system based on machine vision technology has the following beneficial effects:

[0052] 1. Based on the perception system, the visual sensor group and various sensors collect multi-dimensional data information of the hook and lifting scene, establish relevant disturbance items that affect the accuracy and safety of hook removal, integrate various disturbance items to build a hook coefficient model and a pressure coefficient model, obtain the hook coefficient that characterizes the degree of alignment and the pressure coefficient that characterizes the safety of the mounting, adopt a dual-coefficient joint decision-making mechanism to determine whether the conditions for hook removal are met, and comprehensively evaluate the composite risk of hook removal. This breaks through the reliance of traditional mechanical positioning on standard cargo types and operating experience, effectively responds to non-standard scenarios such as stacking deformation of bagged cargo and ship swaying, and can realize intelligent hook removal decision-making in complex environments and multi-dimensional disturbances. The expansion rate of applicable cargo types is high, and attention to the safety of the hook and loading status can improve the safety of hook removal. Therefore, based on this hook removal method, automatic hook removal can be realized during intelligent lifting, breaking the "last mile" breakpoint of the intelligent construction of smart ports;

[0053] 2. The intelligent analysis module uses dual-coefficient, multi-threshold, and combined thresholds to classify unhookable conditions into three levels of early warning responses, building a hierarchical safety defense line and triggering different processing tasks. While taking into account lifting efficiency, it shortens accident early warning response time, improves the safety factor of manual operation, and reduces equipment damage caused by misoperation.

[0054] 3. The storage module uses blockchain distributed evidence storage to ensure that operation logs cannot be tampered with and the entire decision-making process is traceable. An optimization trigger mechanism is set up, and trusted historical data is used to drive threshold self-optimization and coefficient model upgrades. The system's adaptive optimization improves and stabilizes the success rate of unhooking, meeting the port's compliance audit requirements and continuous development needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a flow chart of the method for automatically unhooking cargo disclosed in the present invention;

[0056] Figure 2 The figure is a schematic diagram of the composition of the automatic hook removing system disclosed in the present invention. DETAILED DESCRIPTION

[0057] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only the best embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0058] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute a separate or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0059] This embodiment provides a method for automatically unhooking goods based on machine vision technology. Figure 1 As shown, the following steps are included:

[0060] S1. Data Collection: Based on visual technology and multi-source sensing elements, multi-dimensional data information of the lifting scene is collected in real time. Specifically:

[0061] The three-dimensional coordinates of the hook and eye are collected through visual technology to monitor the three-dimensional position deviation between the hook and the cargo in real time to ensure accurate alignment. Multi-source sensing collects the hook's inclination angle, pressure, load, equipment status, and ambient wind speed.

[0062] The vision technology is based on a binocular camera and a Time of Flight (ToF) depth sensor. Both are mounted on the gantry crane's crossbeam or boom, and aimed at the hook and spreader. Together, they scan and determine the position data of the hook and spreader's lifting rings, monitoring the three-dimensional positional deviation between the hook and the cargo in real time to ensure precise alignment.

[0063] The hook inclination is collected by the inclination sensor. Both the inclination sensor and the gyro IMU are installed at the connection between the hook and the spreader. The inclination sensor collects the real-time swing angle, and the gyro IMU monitors the swing amplitude to prevent unhooking or collision due to shaking.

[0064] The ambient wind speed is collected in real time by a digital anemometer installed on the top of the boom;

[0065] Pressure refers to the contact pressure of the hook, which is measured by a six-dimensional force sensor installed at the connection between the hook and the eye. The six-dimensional force sensor can accurately measure the contact force of the hook under multi-dimensional joint load, preventing the hook from being locked too tight or too loose due to bite failure;

[0066] The pressure sensor is used to collect cargo weight data. It is integrated into the load-bearing frame of the spreader to facilitate real-time calculation of the load factor and avoid overloading risks.

[0067] The monitoring of equipment status is based on the electromagnetic lock hook, which collects current data in real time through the current sensor electrically connected to the electromagnetic lock control circuit, so as to monitor the working status of the electromagnetic lock and reflect mechanical wear or electrical failure;

[0068] S2. Data preprocessing: Establish a dynamic three-dimensional coordinate system based on the real-time three-dimensional position data of the lifting ring, calculate the dynamic position deviation based on the three-dimensional position data of the hook, and calculate the position deviation coefficient of the position deviation disturbance term. The position deviation disturbance term quantifies the alignment accuracy between the hook and the cargo eye. The larger the deviation, the greater the risk of unhooking:

[0069]

[0070] The swing disturbance term suppresses the swing influence through nonlinear attenuation, and the real-time swing angle of the inclination sensor is , calculate the swing coefficient of the swing disturbance term :

[0071]

[0072] Calculate the drag coefficient of the wind resistance disturbance term based on the ambient wind speed :

[0073]

[0074] The position deviation disturbance term, swing disturbance term and wind resistance disturbance term are dynamic factors that focus on spatial alignment and environmental disturbances, among which, is the dynamic deviation of the three-dimensional position of the hook and the cargo ring, that is, ( , , ), is the modulus of the three-dimensional position dynamic deviation measured by the Euclidean norm, is the maximum allowable deviation modulus, defined by engineering safety standards; k is the angle suppression factor, which controls the rate of increase of the index. According to the swing amplitude monitoring of the gyroscope IMU, the stronger the swing, the larger the value assigned, and the swing disturbance coefficient The bigger; is the real-time wind speed, The critical wind speed is exceeded and the operation should be forced to stop;

[0075] Calculate the force balance coefficient of the force balance disturbance term based on the pressure data of the six-dimensional force sensor :

[0076]

[0077] Calculate the load matching coefficient based on the cargo weight of the pressure sensor and the disturbance term :

[0078]

[0079] Calculate the device health coefficient of the device health disturbance item based on the current data of the current sensor :

[0080]

[0081] The force balance disturbance term, load matching disturbance term and equipment health disturbance term are static factors that focus on mechanical safety and equipment status. It is the real-time contact pressure when the hook is lifted. For the best contact pressure, the hook is in a completely stable bite state. The maximum permissible contact pressure is exceeded, and the lock is too tight. The minimum permissible contact pressure is below which the lock is too loose. is the weight of the cargo, The rated load of the hook; is the actual working current of the hook electromagnetic lock, The nominal current of the hook electromagnetic lock is the expected current value when the device is working normally;

[0082] S3. Construct the coefficient model of the fusion-related disturbance terms:

[0083] The initial hook coefficient model is established by integrating the disturbance terms of dynamic factors, at least integrating the above-mentioned position deviation disturbance term, swing disturbance term and wind resistance disturbance term, and calculating the hook coefficient H:

[0084]

[0085] Among them, the hook coefficient H represents the degree of spatial alignment between the hook and the cargo and the environmental stability. The smaller the H value, the better the alignment. , is the weight coefficient of the position deviation disturbance term, reflecting the importance of spatial alignment in the dehooking decision, and is generally assigned a value between 0.5 and 0.6. is the weight coefficient of the swing disturbance term, which is generally assigned a value between 0.3 and 0.4. γ is the weight coefficient of the wind resistance disturbance term, which is generally assigned a value between 0.2 and 0.3, and α+β+γ=1. The values ​​of each weight coefficient can be dynamically adjusted according to the actual situation.

[0086] The disturbance term of static factors is integrated to establish the initial pressure coefficient model, which at least integrates the above-mentioned force balance disturbance term, load matching disturbance term and equipment health disturbance term to calculate the pressure coefficient P:

[0087]

[0088] Among them, the pressure coefficient P represents the contact stability and load matching of the hook. , the smaller the P value is, the safer the contact state is. The weight coefficient of the force balance disturbance term reflects the impact of force balance on the risk of hook removal during locking, and is generally assigned a value between 0.6 and 0.65; The weight coefficient of the load matching disturbance term is generally assigned a value between 0.3 and 0.35; is the weight coefficient of the equipment health disturbance term, which is generally assigned a value between 0.1 and 0.15, and + + =1, the values ​​of each weight coefficient can be dynamically adjusted according to actual conditions;

[0089] S4. Generate instructions: Set the hook coefficient threshold, pressure coefficient threshold, and dual-coefficient joint threshold at each level. Use the dual-coefficient joint decision-making mechanism to determine whether the unhooking conditions are met. If the unhooking conditions are met, send the unhooking instruction. Further analyze and determine the dual-coefficient values ​​that do not meet the unhooking conditions, and send warning signals of different levels. Specifically, the dual-coefficient joint decision-making mechanism includes the following:

[0090] 1) Setting hook coefficient thresholds, including the initial threshold h, and unhooking urgency thresholds h1 and h2; Setting pressure coefficient thresholds, including the initial threshold p, and unhooking urgency thresholds p1 and p2;

[0091] 2) The unhooking condition is H≤h and P≤p. If the unhooking condition is met, the unhooking instruction is sent;

[0092] 3) Warning level: , and H+P≤t; or, , and H+P≤t; if any of the above two warning conditions is met, a warning level signal is sent;

[0093] 4) Intervention level: ;or ; or H+P ; If any of the three intervention conditions mentioned above is met, an intervention level signal is sent;

[0094] 5) Emergency level: H> ; or P> ; or H+P> ; If any of the three emergency conditions mentioned above is met, an emergency level signal will be sent.

[0095] S5. Trigger different processing tasks according to the hook removal instruction or different levels of warning signals:

[0096] 1) The electromagnetic lock control circuit is triggered by the unhooking command, and the current generates an electromagnetic drive to control the unlocking pin to achieve pin extension and retraction. The laser scanner monitors the hook opening angle in real time. If unhooking is blocked, the pneumatic auxiliary push rod is activated to provide additional thrust until the unhooking and unloading are automatically completed. The laser scanner sends a "unhooking completed" status code to the lifting control system and simultaneously updates the cargo status of the terminal dispatch system to "unloaded";

[0097] 2) Triggering enhanced status monitoring based on early warning level signals, that is, enhancing the intensity and accuracy of perception. For example, the binocular camera starts the multi-spectral compensation mode, and the sampling frequency of the binocular camera and ToF depth sensor is increased to 2 times, compressing the measurement error of the tilt sensor. In addition, the operation accuracy is improved. For example, the inertial navigation system carried by the boom can be switched to high-precision mode. At the same time, the sound and light prompts are triggered, and a yellow warning box pops up on the HMI interface of the control room. If there are abnormal parameters, the abnormal parameters are displayed and the working condition parameters are recorded in real time in the blockchain log. Abnormal parameters such as the cargo weight exceeds the rated load and the contact pressure when the hook is locked exceeds wait;

[0098] 3) Trigger the emergency operation mode and manual confirmation process based on the intervention level signal. The emergency operation mode improves operation safety and introduces human intervention for investigation. The operator manually confirms whether to continue the operation. Specifically, it includes the following:

[0099] a. The hook automatically switches to double insurance mode, the main lock remains closed, and the backup mechanical lock is activated; the power system of the lifting equipment automatically reduces power and runs at a speed limited to 30% of the normal value; b. The multi-dimensional perception data diagnostic report is automatically pushed to the lifting control system, including the hook's three-dimensional deviation vector diagram, the hook's contact force change curve within the past 30 seconds, and the electromagnetic lock current spectrum analysis results; c. The operator completes double verification, such as biometric fingerprint + face recognition to confirm identity, and manually circles key risk points in the three-dimensional simulation interface for inspection. If the fault point is locked, the fault handling process is entered. If it is confirmed to be safe, the emergency operating lever is continuously pressed to release the lock and resume normal operation.

[0100] 4) Upon receiving an emergency-level signal, emergency braking is directly executed. The hoisting system autonomously performs deep braking, using a combination of electromagnetic brakes and hydraulic damping. The power source is cut off, the main circuit contactor is forcibly disconnected, and the UPS power supply takes over the power supply for key sensors. The wireless emergency communication channel is activated, bypassing the conventional network and connecting directly to the control center. The mechanical locking device is activated, the hydraulic brake device fully locks the boom, and the emergency counterweight is automatically dropped to quickly reduce the boom torque. An alarm signal is sent to the control center, automatically activating the on-site rotating red warning light and pulse alarm, and the terminal broadcast system automatically broadcasts warnings in both Chinese and English.

[0101] In an exemplary embodiment of the present disclosure, the automatic unhooking method further includes: step S6, coefficient model upgrade and iterative optimization of thresholds: dynamic threshold calibration and algorithm model upgrade are performed based on traceable trusted data evidence to form an automated closed-loop system integrating accurate identification, intelligent decision-making, safety control and self-iteration, significantly improving the safety and efficiency of unhooking operations and enabling intelligent upgrades of ports; the intelligent analysis module sets an optimization trigger mechanism, which includes active optimization, periodic optimization and event-driven optimization; wherein,

[0102] Active optimization includes retraining and upgrading the coefficient model based on incremental data. By introducing transfer learning, the coefficient model verified to be effective in resisting interference at other terminals is migrated to the local coefficient model.

[0103] Periodic optimization is automatically initiated based on a preset operation cycle or preset operation volume. For example, the periodic optimization process is automatically initiated every time 1,000 valid operations are accumulated or every 30 days, whichever comes first.

[0104] Periodic optimization includes setting a dynamically adjusted threshold for the confidence interval of credible data. If the confidence interval is set to 95%, the distribution of the hook coefficient H and pressure coefficient P of successful hook removal in the credible data is extracted, and a reference threshold that satisfies the 95% confidence interval is calculated as the new threshold. For example, if the original hook coefficient threshold H is 0.3, and statistical analysis shows that 95% of successful hook removal cases in actual safe operations have a hook coefficient ≤ 0.35, then the new threshold can be relaxed to 0.35.

[0105] Periodic optimization also includes mining potential correlation factors based on trustworthy data and introducing a composite impact index upgrade coefficient model; for example, mining the coupling relationship between wind speed and swing angle based on trustworthy data, when ≤ When the wind force has a continuous boosting effect on the swing angle, the two show a linear coupling relationship. express, It ensures that the wind force is always in the same direction as the swing amplitude, and introduces the wind energy-mechanical energy conversion efficiency k1. Thus, the swing disturbance term is upgraded by adding the composite impact index of wind force and swing angle:

[0106]

[0107] Event-driven optimization triggers special optimization thresholds based on the number of misjudgment types or special cargo lifting:

[0108] 1) Optimize the misjudgment types that reach a certain number of times by quantifying the loss and rebalancing the threshold using the ROC curve. Specifically, this includes the following:

[0109] For example, if the number of false emergency braking is W, the efficiency loss weight of misjudgment is 1, the number of correct emergency braking is S, the number of accidents caused by missed emergency braking is L, the accident loss weight is 10, and the normal number of times no emergency braking is performed is Z, the loss function of emergency braking is established:

[0110] ;

[0111] Calculate the total loss under each joint threshold t (t), with the horizontal axis X=W / (W+S) and the vertical axis Y=L / (L+H), draw the ROC curve of the total loss and calculate the optimal threshold that minimizes the total loss :

[0112]

[0113] 2) When lifting overweight special cargo, that is, when M>1.2 When the load compensation mechanism is triggered according to the confirmation, a short-term overloading operation is authorized, the original initial threshold value P of the pressure coefficient is compensated, and the overweight cargo pressure coefficient threshold value P' under the overloading operation is generated:

[0114]

[0115] In an exemplary embodiment of the present disclosure, the automatic hook removal method also considers interference from other working environments besides ambient wind speed. Environmental interference reduces perception accuracy, and the method achieves disturbance term adaptation by increasing the environmental adaptation weight, which includes the following:

[0116] 1) Detect rainfall and rainfall intensity In order to compensate for the interference of rain on the visual sensor group, the weight coefficient of the position deviation disturbance term is increased to generate an adaptive weight coefficient ':

[0117]

[0118] 2) Haze weather is detected, is the minimum visibility, take 100m, For normal visibility, take 200m. When visibility V is less than 100m, in order to compensate for the decrease in visual positioning accuracy, the weight coefficient of wind resistance disturbance term is increased to generate adaptive weight coefficient ':

[0119]

[0120] It is understandable that the linkage coefficient model after adaptive weight adjustment still meets the distribution requirement that the sum of weight coefficients is 1, and the weight coefficients that have not been adaptively adjusted are adaptively reduced or the adaptation ratio is reduced.

[0121] This embodiment also provides a system for automatically unhooking cargo hooks. Figure 2 As shown, intelligent hook removal is achieved based on the above hook removal method, wherein the hooks mentioned in the present invention all refer to automatic unlocking hooks, which are provided with dual locks, the main lock is an electromagnetic lock, and the backup lock is a mechanical lock. The automatic hook removal system is connected to the lifting control system to feedback monitoring data, two-way feedback decision data and control instructions, etc. It includes a perception system, an intelligent analysis module, a hierarchical execution control module and a storage module; wherein,

[0122] The perception system is used for monitoring and intelligent control of the hook removal process and data source collection, including a visual sensor group, an inclination sensor, a gyroscope IMU, a digital anemometer, a six-dimensional force sensor, a pressure sensor group, a current sensor, and a laser scanner. The visual sensor group includes a binocular camera and a ToF depth sensor, which are used to scan in real time and jointly determine the spatial position coordinate data of the hook and the sling ring. The inclination sensor collects the swing angle of the sling, the gyroscope IMU determines the swing amplitude of the cargo, the digital anemometer collects real-time wind speed data, the six-dimensional force sensor collects the contact pressure data between the cargo hook and the sling ring when the hook is engaged, the pressure sensor group collects cargo weight data, the current sensor is connected to the electromagnetic lock control circuit to collect electromagnetic lock current data, and the laser scanner captures the hook opening angle during the unlocking and locking process of the hook. When the unlocking is completed, the "unloading completion" data is fed back to the lifting control system, and when the locking is completed, the "hanging completion" data is fed back to the lifting control system.

[0123] The intelligent analysis module includes a disturbance item calculation unit, a coefficient fusion calculation unit, a decision unit, and an optimization unit. The disturbance item calculation unit calculates various initial disturbance items or adaptive disturbance items under interference environment based on the data collected by the perception system. The coefficient fusion calculation unit calculates the hook coefficient and pressure coefficient based on the preset or optimized weight coefficient and coefficient model of the optimization unit. The decision unit presets the threshold and generates the corresponding control instruction based on the calculation result of the coefficient fusion calculation unit and sends it to the hierarchical execution control module. The optimization unit optimizes the weight coefficient or coefficient model based on the real-time data, historical data, and incremental data in the storage module, triggered by the optimization trigger mechanism, and feeds back to the coefficient fusion calculation unit.

[0124] The hierarchical execution control module includes a hierarchical control end and a hierarchical execution end. The hierarchical control end receives the decision instructions of the decision unit, generates corresponding control signals and sends them to each node of the hierarchical execution end and the lifting control system. The hierarchical execution end includes an electromagnetic lock, a mechanical lock, a perception system and a manual inspection and guidance system; the hierarchical control end triggers the electromagnetic lock to unlock according to the hook-removing instruction of the decision unit; triggers the perception system to strengthen monitoring according to the early warning level instruction, and simultaneously sends it to the lifting control system to improve the operation accuracy of the boom, and pops up windows and sound and light warnings for abnormal parameters; triggers the emergency operation mode and the manual inspection and guidance system according to the intervention level instruction to improve the safety of operations under abnormal conditions, calls out the perception data of key risk points, introduces human intervention, and the operator selects key risk points for inspection and confirmation, eliminates the fault and considers the lock released; sends an emergency braking signal to the lifting control system according to the emergency level instruction, including cutting off the power supply, deep braking of the lifting system, locking of the boom and sound and light warnings.

[0125] In an exemplary embodiment of the present disclosure, in order to continuously optimize the algorithm model and threshold parameters using storable data, the storage module uses blockchain to store credible data logs to ensure that the operation logs cannot be tampered with, the entire decision-making process is traceable, and the historical data source is reliable. This meets the requirements of port compliance audits, and the system's adaptive optimization improves and stabilizes the success rate of unhooking. Each round of operation generates a data block with a log structure. The blockchain log structure includes:

[0126] Operation fingerprint: timestamp, spreader ID, cargo type, environmental label, etc.

[0127] Raw data from the perception system: visual positioning deviation, hook pressure value, electromagnetic lock current waveform fragment, etc.

[0128] Decision-making process data: double coefficient calculation value, threshold determination result, execution action, etc.

[0129] Manual correction records: identification information of the operator when they intervene, parameter adjustment content, and reasons, etc.

[0130] In addition, the storage module adopts a consortium chain architecture, with terminal managers, equipment manufacturers, and regulatory agencies as consensus nodes, providing a complete chain of evidence for accident tracing, solving the problem of responsibility definition in traditional black box systems; accumulating high-quality scene data, and avoiding dirty data pollution caused by sensor noise or communication packet loss.

[0131] Through the description of the above embodiments, those skilled in the art can clearly understand that the various embodiments of the present application can be implemented by means of software or software combined with a necessary general hardware platform, and of course can also be implemented by hardware functions. Based on such understanding, the technical solution of the present application can essentially be embodied in the form of a software product or the part that contributes to the prior art. The software product is stored in a storage medium and includes a number of instructions for enabling a computer device, such as but not limited to a personal computer, a server, or a network device, to execute all or part of the steps of the method described in any embodiment of the present application.

[0132] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for automatically unhooking goods based on machine vision technology, characterized in that: The following steps are involved: S1. Data Collection: The three-dimensional coordinates of the hook and the eye are collected using visual technology, and hook-related data and environmental data are collected using multi-source perception. The hook-related data includes the hook's inclination angle, pressure, load, and device status, and the environmental data includes ambient wind speed. S2. Data preprocessing: establishing a dynamic three-dimensional coordinate system based on the three-dimensional position coordinates of the lifting ring, calculating the position deviation of the hook, and establishing at least a position deviation disturbance term, a swing disturbance term, a wind resistance disturbance term, a force balance disturbance term, a load matching disturbance term, and an equipment health disturbance term based on the hook-related data and the environment-related data; S3. Constructing a coefficient model integrating related disturbance terms: integrating at least the position deviation disturbance term, the swing disturbance term, and the wind resistance disturbance term to establish a hook coefficient model and calculate the hook coefficient; integrating at least the force balance disturbance term, the load matching disturbance term, and the equipment health disturbance term to establish a pressure coefficient model and calculate the pressure coefficient; S4. Generate instructions: A dual-coefficient joint decision-making mechanism is used to make decoupling decisions. Hook coefficient thresholds, pressure coefficient thresholds, and dual-coefficient joint thresholds are set at each level. If the hook coefficient and the pressure coefficient meet the decoupling conditions, a decoupling instruction is issued; if they do not meet the decoupling conditions, a warning signal is issued. S5. Trigger different processing tasks according to the hook removal instruction or the early warning signal.

2. The automatic cargo hook removal method based on machine vision technology according to claim 1 is characterized in that: It also includes the upgrade of the coefficient model and iterative optimization of thresholds, and the setting of optimization trigger mechanisms, including active optimization, periodic optimization, and event-driven optimization; The active optimization fine-tunes the coefficient model based on incremental data and introduces transfer learning to migrate the off-site coefficient model that has been verified to be effective in resisting interference at other terminals to the local terminal for optimization; The periodic optimization includes: setting a dynamic adjustment threshold of the confidence interval of credible data according to a preset operation cycle or a preset operation volume, and mining potential correlation factors based on the credible data, introducing composite influence indicators to upgrade the coefficient model; the event-driven optimization triggers special optimization according to the number of misjudgment types or special cargo lifting.

3. The automatic cargo hook removal method based on machine vision technology according to claim 2 is characterized in that: The hook coefficient model is: The linkage coefficient ; Position deviation coefficient ; Swing coefficient ; Drag compensation coefficient ; in, 、 and are the weight coefficients of the position deviation disturbance term, the swing disturbance term and the wind resistance disturbance term, respectively, and + + =1; is the three-dimensional position deviation between the hook and the ring, that is, ( , , ), using the Euclidean norm The modulus length that measures the three-dimensional position deviation, is the maximum allowable deviation modulus; is the swing angle of the hook, k is the angle suppression factor; is the real-time wind speed, is the critical wind speed.

4. The automatic cargo hook removal method based on machine vision technology according to claim 3 is characterized in that: The pressure coefficient model is: The pressure coefficient ; Force balance coefficient ; Load matching coefficient ; Equipment health coefficient ; in, 、 and are the weight coefficients of the force balance disturbance term, the load matching disturbance term, and the equipment health disturbance term, respectively, and + + =1; is the real-time contact pressure of the hook, For optimal contact pressure, is the maximum allowable contact pressure, is the minimum permissible contact pressure; is the weight of the cargo, is the rated load of the hook; is the actual working current of the hook electromagnetic lock, is the nominal current of the hook electromagnetic lock.

5. The automatic cargo hook removal method based on machine vision technology according to claim 2 is characterized in that: When other working environment interferences other than the ambient wind speed occur, the environmental adaptation weight is increased, including the following: 1) Detect rainfall and rainfall intensity When the weight coefficient of the position deviation disturbance term is increased, an adaptive weight coefficient is generated. ': ； 2) Haze weather is detected and visibility V < , increase the weight coefficient of the wind resistance disturbance term to generate an adaptive weight coefficient ': ； in, For minimum visibility, Normal visibility.

6. The automatic cargo hook removal method based on machine vision technology according to claim 4 is characterized in that: The unhooking threshold h of the hook coefficient and the unhooking threshold p of the pressure coefficient are preset. When H≤h and P≤p, the unhooking condition is met and the unhooking instruction is sent. For situations where the unhooking instruction is not met, at least three warning levels are set to generate different decision instructions. The unhooking urgency thresholds h1 and h2 of the hook coefficient, the unhooking urgency thresholds p1 and p2 of the pressure coefficient, and the dual-coefficient joint thresholds t and T are preset. The warning level is set as: Warning level: , and H+P≤t; or, , and H+P≤t; triggering state monitoring enhancement to improve operation accuracy; Intervention level: ;or ; or H+P ;Trigger emergency operation mode and manual confirmation process; Emergency level: H> ; or P> ; or H+P> ; trigger emergency braking, cut power and send alarm.

7. The automatic cargo hook removal method based on machine vision technology according to claim 2 is characterized in that: For the misjudgment type that reaches a certain number of times, the loss is quantified, and the threshold is rebalanced through the ROC curve. The optimization balance process includes the following: Define the number of misjudgments of a certain type of dangerous event as W, and the efficiency loss weight of misjudgment as , the number of correct identification and triggering is S, the number of missed reports is L, and the accident loss weight of missed reports is , the normal number of times without false triggering is H, and the loss function is established: ； Calculate the total loss at each threshold t (t), with the horizontal axis X=W / (W+S) and the vertical axis Y=L / (L+H), create an ROC curve and calculate the optimal threshold that minimizes the total loss : 。 8. The method for automatically unhooking cargo based on machine vision technology according to claim 4, characterized in that: For overweight special cargo, i.e. M> hour, >1, the load compensation mechanism is triggered, and the overweight cargo pressure coefficient threshold P' is used to allow short-term overload operations: 。 9. An automatic cargo hook removal system, based on the automatic cargo hook removal method based on machine vision technology according to any one of claims 1 to 8, characterized in that: The hook is an automatic unlocking hook with a double lock, wherein the main lock is an electromagnetic lock and the backup lock is a mechanical lock. The automatic hook removal system is connected to the lifting control system, which includes a perception system, an intelligent analysis module, a hierarchical execution control module and a storage module; wherein, The perception system includes a visual sensor group, an inclination sensor, a gyroscope IMU, a digital anemometer, a six-dimensional force sensor, a pressure sensor group, a current sensor and a laser scanner; the visual sensor group includes a binocular camera and a ToF depth sensor arranged at the front end of the gantry crane beam or boom, for real-time scanning of the three-dimensional position data of the hook and the lifting ring; the inclination sensor and the gyroscope IMU are arranged at the connection between the hook and the spreader, for collecting and monitoring the swing angle and amplitude of the spreader; the digital anemometer is installed at the top of the boom, for collecting real-time wind speed data; the six-dimensional force sensor is installed at the connection between the hook and the lifting ring, for collecting the contact pressure data of the hook; the pressure sensor group is built into the spreader, for collecting cargo weight data; the current sensor is connected to the electromagnetic lock control circuit, for collecting the current data of the electromagnetic lock; the laser scanner is arranged on the hook, for capturing the unlocking and locking process of the hook; The intelligent analysis module includes a disturbance term calculation unit, a coefficient fusion calculation unit, a decision unit, and an optimization unit. The disturbance term calculation unit calculates various disturbance terms based on the data collected by the perception system. The coefficient fusion calculation unit calculates the hook coefficient and the pressure coefficient based on the preset weight coefficient and coefficient model or the weight coefficient optimized by the optimization unit. The decision unit analyzes the result of the coefficient fusion calculation unit according to a preset threshold and generates corresponding control instructions. The optimization unit triggers the optimization of the weight coefficient or the coefficient model according to the optimization trigger mechanism based on the real-time data, historical data, and incremental data in the storage module. The hierarchical execution control module includes a hierarchical control end and a hierarchical execution end. The hierarchical execution end includes the electromagnetic lock, the mechanical lock, the perception system and the manual inspection and guidance system. The hierarchical control end triggers the electromagnetic lock to unlock according to the unhooking instruction of the decision unit; triggers the perception system to strengthen monitoring according to the early warning level instruction, and sends a signal to improve operation accuracy to the lifting control system; triggers the manual inspection and guidance system according to the intervention level instruction, activates the backup lock and sends an emergency operation mode signal to the lifting control system to improve the safety of abnormal operations and introduce human intervention; sends an emergency braking signal to the lifting control system according to the emergency level instruction.

10. The automatic cargo hook removal system according to claim 9, characterized in that: The storage module uses a blockchain log structure to store trusted data and adopts a consortium chain architecture, with terminal managers, equipment manufacturers, and regulatory agencies as consensus nodes. The blockchain log structure includes: Operation fingerprint: including timestamp, spreader ID, cargo type, and environmental tag; Raw data from the perception system: including visual positioning deviation, hook pressure value, and electromagnetic lock current waveform fragment; Decision-making process data: including double coefficient calculation values, threshold determination results, and execution actions; Manual correction records: include parameter adjustment contents and reasons when the operator intervenes.

Images