Method and system for automatic hooking and unhooking of goods based on machine vision technology

Abstract

The application discloses a cargo automatic unhooking method and system based on machine vision technology, which comprises data acquisition; data preprocessing, wherein disturbance terms of various factors are respectively established according to various relevant data and environment relevant data of hooks; a hook coefficient model and a pressure coefficient model fusing the relevant disturbance terms are constructed; instructions are generated; hook coefficient threshold values, pressure coefficient threshold values and double-coefficient joint threshold values are set, a double-coefficient joint decision mechanism is used to judge whether the unhooking condition is met, and a warning signal is sent if the unhooking condition is not met; different processing tasks are triggered according to the unhooking instructions or the warning signal; the method and the system can comprehensively evaluate unhooking composite risks, break through the dependence of traditional mechanical positioning on standard cargo types and operation experience, realize intelligent unhooking decision under complex environment and multi-dimensional disturbance, have high cargo type expansion rate, and guarantee unhooking safety; and the unhooking method can realize automatic unhooking and break through the "last one kilometer" breakpoint of port intelligent construction.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of automated hoisting technology, in particular to a cargo automatic hooking and unhooking method and system based on machine vision technology. BACKGROUND

[0002] With the container terminal automation rate increasing to more than 90%, the terminal hoisting and transportation links have been realized unmanned, but the hooking and unhooking links are still not intelligent enough, and still rely on manual intervention in complex environments, becoming the "last mile" breakpoint of smart port construction.

[0003] Manual hooking is difficult, risky and has insufficient operating environment safety, and traditional manual operation does not meet the development needs of intelligent production and transportation; by physically isolating personnel from the dangerous operation area through semi-automatic equipment and remotely controlling and overall judging by manual operation, the safety and efficiency of traditional manual operation are solved to some extent, but there are still some limitations, one is that the environmental adaptability and positioning accuracy have structural defects, and the existing mechanical positioning system mostly relies on rigid structure to match standard cargo types, and it is difficult to maintain stable alignment in non-standard scenes such as bagged cargo stacking deformation and ship swaying, and the subjective perception of remote operation to complex cargo environments excessively depends on experience, and cannot perform spatial alignment under dynamic changes and environmental disturbances, such as cargo position deviation, swinging conditions, environmental wind disturbance and other disturbance factors affecting the accuracy of hooking, and the cargo type expansion rate is low; second, the device state and mechanical safety are not monitored, compared with traditional manual hooking, remote operators cannot timely understand the safety and state of the hook, and only monitoring mechanical parameters cannot judge the combined risks, such as being unable to grasp the hook locking state, hook health state and load rate and other factors affecting successful hooking; third, the safety warning mechanism is simple, the coupling degree of manual checking link and equipment control is low, the single-dimensional threshold determination mechanism is difficult to capture the combined risks in complex working conditions, the accident warning is single, the response time is long, and the false triggering rate is high, and the delay of emergency braking execution easily leads to an increase in equipment damage rate.

[0004] Therefore, a cargo automatic hooking and unhooking method and system based on machine vision technology are needed to solve the above problems. SUMMARY

[0005] The present application aims to overcome the defects of the prior art and provide a cargo automatic hooking and unhooking method and system based on machine vision technology to solve the problems raised in the background.

[0006] In order to achieve the above-mentioned purpose, the technical scheme of the present application is as follows:

[0007] A cargo automatic hooking and unhooking method based on machine vision technology, comprising the following steps:

[0008] S1, data collection: collect the three-dimensional position coordinates of the hook and the sling based on visual technology, collect hook-related data and environment-related data based on multi-source perception, the hook-related data includes the inclination, pressure, load and equipment state of the hook, and the environment-related data includes the environmental wind speed;

[0009] S2, data preprocessing: establish a dynamic three-dimensional coordinate system according to the three-dimensional position coordinates of the sling, calculate the position deviation of the hook, and at least establish a position deviation disturbance term, a swing disturbance term, a wind resistance disturbance term, a force balance disturbance term, a load matching degree disturbance term and a device health degree disturbance term according to the hook-related data and the environment-related data;

[0010] S3, construct a coefficient model that fuses related disturbance terms: at least fuse the position deviation disturbance term, the swing disturbance term and the wind resistance disturbance term to establish a hook coefficient model and calculate the hook coefficient, and at least fuse the force balance disturbance term, the load matching degree disturbance term and the device health degree disturbance term to establish a pressure coefficient model and calculate the pressure coefficient;

[0011] S4, generate instructions: adopt a double-coefficient joint decision mechanism to make a hook removal decision, set hook coefficient threshold values, pressure coefficient threshold values and double-coefficient joint threshold values, send a hook removal instruction if the hook coefficient and the pressure coefficient meet the hook removal conditions, and send a warning signal if the hook removal conditions are not met;

[0012] S5, trigger different processing tasks according to the hook removal instruction or the warning signal.

[0013] Further, it also includes coefficient model upgrading and threshold iteration optimization, sets an optimization triggering mechanism, including active optimization, periodic optimization and event-driven optimization; the active optimization adjusts the coefficient model based on incremental data, introduces transfer learning, and migrates other wharf verified effective anti-interference off-site coefficient model to the local for optimization; the periodic optimization includes: setting a confidence interval dynamic adjustment threshold for the confidence data according to a preset operation period or a preset operation amount, and introducing a composite influence index to upgrade the coefficient model according to the potential correlation factors mined from the confidence data; the event-driven optimization triggers special optimization according to the number of misjudgment types or the hoisting of special goods.

[0014] Further, the hook coefficient model is:

[0015] Hook coefficient ;

[0016] Position deviation coefficient ;

[0017] Swing coefficient ;

[0018] Wind resistance compensation coefficient ;

[0019] wherein, , and are the weight coefficients of the position deviation disturbance term, the swing disturbance term and the wind resistance disturbance term respectively, and + + = 1; is the three-dimensional position deviation of the hook and the cargo sling ring, i.e. , , , the modulus of the three-dimensional position deviation is measured by the Euclidean norm , is the maximum allowed deviation modulus; is the hook swing angle, and k is the angle suppression factor; is the real-time wind speed, is the critical wind speed.

[0020] Further, the pressure coefficient model is:

[0021] the pressure coefficient ;

[0022] the force balance coefficient ;

[0023] the load matching degree coefficient ;

[0024] the equipment health degree coefficient ;

[0025] wherein, , and are the weight coefficients of the force balance disturbance term, the load matching degree disturbance term and the equipment health degree disturbance term respectively, and + + = 1; is the real-time contact pressure of the hook, is the optimal contact pressure, is the maximum allowed contact pressure, is the minimum allowed contact pressure; is the weight of the cargo, is the rated load of the hook; is the actual working current of the hook electromagnetic lock, is the nominal current of the hook electromagnetic lock.

[0026] Further, when other working environment disturbances other than the environmental wind speed occur, the environmental self-adaptation weight is increased, including the following contents:

[0027] 1) Detecting rainfall and rainfall intensity When the wind speed is greater than the preset threshold, increase the weight coefficient of the position deviation disturbance term to generate an adaptive weight coefficient

[0028]

[0029] 2) Detecting haze weather, and visibility V and visibility V < V Increase the weight coefficient of the wind resistance disturbance term to generate an adaptive weight coefficient

[0030]

[0031] Wherein, The minimum visibility, The normal visibility.

[0032] Further, the hooking threshold h of the preset hooking coefficient and the hooking threshold p of the pressure coefficient are set, when H≤h and P≤p, the hooking condition is met, and the hooking instruction is sent; At least three warning levels that do not meet the hooking are set, different decision instructions are generated, the hooking emergency threshold h1 and h2 of the hooking coefficient and the hooking emergency threshold p1 and p2 of the pressure coefficient are set, and the joint threshold t and T of the two coefficients are set. The warning level is set as:

[0033] Warning level: , and H+P≤t; or, , and H+P≤t; Trigger state monitoring enhancement to improve operation accuracy;

[0034] Intervention level: ; or ; or H+P ; Trigger emergency operation mode and manual confirmation process;

[0035] Emergency level: H> ; or P> ; or H+P> ; Trigger emergency braking, power cut-off and send alarm.

[0036] Further, the loss quantification is performed on the misjudgment type reaching the number of times, and the threshold is rebalanced through the ROC curve, and the optimization and balance process includes the following contents: define the misjudgment number of a certain type of dangerous event as W, the efficiency loss weight of misjudgment , the correct recognition and trigger number is S, the missing report number is L, the missing report accident loss weight is , the normal number of non-mis-trigger is Z, and the loss function is established:

[0037]

[0038] Calculate the total loss under each threshold t​​​​​ (t), to make ROC curve with horizontal axis X = W / (W+S), vertical axis Y = L / (L+H), calculate the optimal threshold value when the total loss is minimum :

[0039] .

[0040] Further, for the special cargo of overweight, that is, M > , > 1, the load compensation mechanism is triggered, the overload operation is allowed in a short period by using the overload cargo pressure coefficient threshold P':

[0041] .

[0042] An automatic hooking and unhooking system for cargo, the hook of which is an automatic unlocking hook provided with double locks, wherein the main lock is an electromagnetic lock and the standby lock is a mechanical lock, the automatic hooking and unhooking system is connected to a hoisting general control system, and comprises a sensing system, an intelligent analysis module, a hierarchical execution control module and a storage module; wherein

[0043] The sensing system comprises a visual sensor group, an inclination sensor, an IMU gyroscope, a digital anemograph, a six-dimensional force sensor, a pressure sensor group, a current sensor and a laser scanner; the visual sensor group comprises a binocular camera and a ToF depth sensor arranged at the front end of a gantry crane beam or a hoisting arm, for scanning the three-dimensional position data of the hook and the lifting ring in real time, the inclination sensor and the IMU gyroscope are arranged at the connection between the hook and the lifting appliance, for collecting and monitoring the swing angle and amplitude of the lifting appliance, the digital anemograph is installed at the top end of the hoisting arm, for collecting real-time wind speed data, the six-dimensional force sensor is installed at the connection between the hook and the lifting ring, for collecting hook contact pressure data, the pressure sensor group is built-in on the lifting appliance, for collecting cargo weight data, the current sensor is connected to the electromagnetic lock control circuit, for collecting current data of the electromagnetic lock, and the laser scanner is arranged on the hook, for capturing the unlocking and locking process of the hook;

[0044] The intelligent analysis module comprises a disturbance term calculation unit, a coefficient fusion calculation unit, a decision unit and an optimization unit, the disturbance term calculation unit calculates various disturbance terms according to the data collected by the sensing system, the coefficient fusion calculation unit calculates the hook coefficient and the pressure coefficient according to the weight coefficient optimized by the optimization unit and the coefficient model, the decision unit analyzes the results of the coefficient fusion calculation unit according to the preset threshold value and generates corresponding control instructions, and the optimization unit triggers the optimization weight coefficient or the coefficient model according to the real-time data, historical data and incremental data in the storage module and according to the optimization trigger mechanism;

[0045] The hierarchical execution control module comprises a hierarchical control end and a hierarchical execution end, the hierarchical execution end comprises an electromagnetic lock, a mechanical lock, a sensing system and a manual troubleshooting guide system, the hierarchical control end triggers the electromagnetic lock to be unlocked according to the unhooking instruction of the decision unit; the sensing system is triggered to strengthen monitoring according to the early warning level instruction, and a signal for improving operation precision is sent to the hoisting general control system; the manual troubleshooting guide system is triggered and the standby lock catch is activated according to the intervention level instruction, and an emergency operation mode signal is sent to the hoisting general control system, so as to improve the abnormal operation safety and introduce human intervention; the signal is sent to the hoisting general control system according to the emergency level.

[0046] Further, the storage module stores the data through a blockchain log structure, and adopts a consortium chain architecture, taking the terminal management party, the equipment manufacturer and the regulatory authority as consensus nodes, and the blockchain log structure comprises:

[0047] ‌Operation fingerprint: including timestamp, spreader ID, cargo type, environment label;

[0048] Sensing system raw data: including visual positioning deviation, hook pressure value, electromagnetic lock current waveform segment;

[0049] ‌Decision process data: including double coefficient calculation value, threshold determination result, execution action;

[0050] ‌Manual correction record: including parameter adjustment content and reason note when the operator intervenes.

[0051] Compared with the prior art, the cargo automatic unhooking method and system based on the machine vision technology have the following beneficial effects:

[0052] 1. The visual sensor group of the sensing system and various sensors collect multi-dimensional data information of the hook and the hoisting scene, establish related disturbance terms affecting the unhooking accuracy and safety, fuse various disturbance terms to build a hook coefficient model and a pressure coefficient model, obtain a hook coefficient representing the alignment degree and a pressure coefficient representing the loading safety, adopt a double coefficient joint decision mechanism to judge whether the unhooking condition is met, comprehensively evaluate the unhooking composite risk, break through the dependence of traditional mechanical positioning on standard cargo types and operation experience, effectively cope with non-standard scenes such as bagged cargo stacking deformation and ship sway, can realize intelligent unhooking decision under complex environment and multi-dimensional disturbance, has high cargo type expansion rate, pays attention to the safety of the hook loading state, and improves the unhooking safety, thereby, the automatic unhooking in the intelligent hoisting process can be realized based on the unhooking method, and the "last kilometer" breakpoint of the intelligent construction of the smart port is broken through;

[0053] 2. The intelligent analysis module uses dual-coefficient, multi-threshold, and combined thresholds to classify unhookable conditions into three levels of early warning responses, building a hierarchical safety defense line and triggering different processing tasks. While taking into account lifting efficiency, it shortens accident early warning response time, improves the safety factor of manual operation, and reduces equipment damage caused by misoperation.

[0054] 3. The storage module uses blockchain distributed evidence storage to ensure that operation logs cannot be tampered with and the entire decision-making process is traceable. An optimization trigger mechanism is set up, and trusted historical data is used to drive threshold self-optimization and coefficient model upgrades. The system's adaptive optimization improves and stabilizes the success rate of unhooking, meeting the port's compliance audit requirements and continuous development needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a flow chart of the method for automatically unhooking cargo disclosed in the present invention;

[0056] Figure 2 The figure is a schematic diagram of the composition of the automatic hook removing system disclosed in the present invention. DETAILED DESCRIPTION

[0057] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only the best embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0058] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute a separate or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0059] This embodiment provides a method for automatically unhooking goods based on machine vision technology. Figure 1 As shown, the following steps are included:

[0060] S1. Data Collection: Based on visual technology and multi-source sensing elements, multi-dimensional data information of the lifting scene is collected in real time. Specifically:

[0061] The three-dimensional coordinates of the hook and eye are collected through visual technology to monitor the three-dimensional position deviation between the hook and the cargo in real time to ensure accurate alignment. Multi-source sensing collects the hook's inclination angle, pressure, load, equipment status, and ambient wind speed.

[0062] The visual technology is based on binocular cameras and ToF depth sensors, both of which are installed on the gantry crane beam or the front end of the boom and are aligned with the hook and the spreader. The two jointly scan and determine the hook and spreader ring position data, which are used to monitor the three-dimensional position deviation of the hook and the cargo in real time, ensuring accurate alignment.

[0063] The hook inclination angle is collected by an inclination sensor, and the inclination sensor and the gyroscope IMU are both installed at the connection between the hook and the spreader. The inclination sensor collects the real-time swing angle, and the gyroscope IMU monitors the swing amplitude to prevent unhooking or collision caused by shaking.

[0064] The environmental wind speed is collected in real time by a digital anemometer installed at the top of the boom.

[0065] The pressure refers to the contact pressure of the hook, which is measured by a six-axis force sensor installed at the connection between the hook and the ring. The six-axis force sensor can accurately measure the contact force under the combined load of multiple dimensions of the hook, preventing the locking failure caused by the hook being too tight or too loose.

[0066] The pressure sensor is used to collect cargo weight data, which is integrated into the spreader load-bearing frame, facilitating real-time load rate calculation and avoiding overload risk.

[0067] The monitoring of the equipment state is based on the electromagnetic lock hook, which collects current data in real time through the current sensor connected to the electromagnetic lock control circuit, facilitating the monitoring of the electromagnetic lock working state and reflecting the mechanical wear or electrical failure.

[0068] S2, data preprocessing: establish a dynamic three-dimensional coordinate system according to the real-time ring three-dimensional position data, calculate the dynamic position deviation combined with the three-dimensional position data of the hook, and calculate the position deviation coefficient of the position deviation disturbance term The position deviation disturbance term quantifies the alignment accuracy of the hook and the cargo ring. The greater the deviation, the greater the risk of unhooking:

[0069]

[0070] The swing disturbance term suppresses the swing influence through nonlinear attenuation, and the swing coefficient of the swing disturbance term is calculated according to the real-time swing angle of the inclination sensor :

[0071]

[0072] The wind resistance coefficient of the wind resistance disturbance term is calculated according to the environmental wind speed :

[0073]

[0074] ​Position deviation disturbance term, swing disturbance term and wind resistance disturbance term are dynamic factors of focusing on space alignment and environmental disturbance, wherein, is the three-dimensional position dynamic deviation of the hook and the cargo lifting ring, i.e. , , ), is the modulus of the three-dimensional position dynamic deviation measured by the Euclidean norm, is the maximum allowed deviation modulus, which is defined by the engineering safety standard; k is an angle suppression factor, which controls the increasing speed of the index, and the stronger the swing is, the larger the value is, and the swing disturbance coefficient is larger; is the real-time wind speed, is the critical wind speed, which should be forced to stop working when it exceeds the critical wind speed;

[0075] The force balance coefficient of the force balance disturbance term is calculated according to the pressure data of the six-dimensional force sensor :

[0076]

[0077] The load matching degree coefficient of the load matching degree disturbance term is calculated according to the cargo load of the pressure sensor :

[0078]

[0079] The equipment health degree coefficient of the equipment health degree disturbance term is calculated according to the current data of the current sensor :

[0080]

[0081] The force balance disturbance term, the load matching degree disturbance term and the equipment health degree disturbance term are static factors of focusing on mechanical safety and equipment state, wherein, is the real-time contact pressure when the hook is lifted, is the optimal contact pressure, at which the hook is in a completely stable engagement state, is the maximum allowed contact pressure, which is too tight if it exceeds the value, is the minimum allowed contact pressure, which is too loose if it is lower than the value; is the weight of the cargo, is the rated load of the hook; is the actual working current of the electromagnetic lock of the hook, is the nominal current of the electromagnetic lock of the hook, i.e. the expected current value when the equipment is working normally;

[0082] S3, construct a coefficient model of the fused related disturbance terms:

[0083] The disturbance term fusing dynamic factors establishes an initial hook coefficient model, at least fusing the above-mentioned position deviation disturbance term, swing disturbance term and wind resistance disturbance term, and calculates a hook coefficient H:

[0084]

[0085] The hook coefficient H represents the spatial alignment degree of the hook and the goods and the environmental stability, and the smaller the value of H is, the better the alignment state is, , is a weight coefficient of the position deviation disturbance term, reflecting the importance of spatial alignment in the hooking decision, and is generally assigned a value between 0.5 and 0.6, is a weight coefficient of the swing disturbance term, and is generally assigned a value between 0.3 and 0.4, and γ is a weight coefficient of the wind resistance disturbance term, and is generally assigned a value between 0.2 and 0.3, and α+β+γ=1, and the assignment of the weight coefficients of each term can be dynamically adjusted according to actual conditions;

[0086] The disturbance term fusing static factors establishes an initial pressure coefficient model, at least fusing the above-mentioned force balance disturbance term, load matching degree disturbance term and equipment health degree disturbance term, and calculates a pressure coefficient P:

[0087]

[0088] The pressure coefficient P represents the hook contact stability and the load matching degree, , and the smaller the value of P is, the safer the contact state is, is a weight coefficient of the force balance disturbance term, reflecting the influence of force balance on the hooking risk when locking, and is generally assigned a value between 0.6 and 0.65; is a weight coefficient of the load matching disturbance term, and is generally assigned a value between 0.3 and 0.35; is a weight coefficient of the equipment health degree disturbance term, and is generally assigned a value between 0.1 and 0.15, and + + =1, and the assignment of the weight coefficients of each term can be dynamically adjusted according to actual conditions;

[0089] S4, generating instructions: setting hook coefficient threshold values at each level, pressure coefficient threshold values and double coefficient joint threshold values, using a double coefficient joint decision mechanism to judge whether the hooking conditions are met, sending a hooking instruction if the hooking conditions are met, further analyzing and judging the double coefficient values if the hooking conditions are not met, and sending different levels of warning signals, specifically, the double coefficient joint decision mechanism includes the following contents:

[0090] 1) setting hook coefficient threshold values, including initial threshold value h, hooking emergency degree threshold values h1 and h2; setting pressure coefficient threshold values including initial threshold value p, hooking emergency degree threshold values p1 and p2;

[0091] 2) Hooking condition is H≤h, and P≤p, if the hooking condition is met, send the unhooking instruction;

[0092] 3) Warning level: , and H+P≤t; or, , and H+P≤t; if any of the above two warning conditions is met, send the warning level signal;

[0093] 4) Intervention level: ; or ; or H+P ; if any of the above three intervention conditions is met, send the intervention level signal;

[0094] 5) Emergency level: H> ; or P> ; or H+P> ; if any of the above three emergency conditions is met, send the emergency level signal.

[0095] S5, according to the unhooking instruction or different levels of warning signals trigger different processing tasks:

[0096] 1) According to the unhooking instruction to trigger the through electromagnetic lock control circuit, through the current generated electromagnetic drive unlocking pin control to realize the pin extension and contraction, the laser scanner monitors the hook opening angle in real time, if the unhooking is blocked, the pneumatic auxiliary push rod is started to provide additional thrust, until the automatic unhooking and unloading is completed, the laser scanner sends the "unhooking completion" status code to the hoisting control system, and synchronously updates the cargo state of the wharf scheduling system to "unloaded";

[0097] 2) According to the warning level signal to trigger state monitoring enhancement, that is, to enhance the sensing intensity and accuracy, for example, the binocular camera starts the multispectral compensation mode, the sampling frequency of binocular camera and ToF depth sensor is increased to 2 times, and the measurement error of compression angle sensor is reduced; In addition, improve the operation accuracy, such as the inertial navigation system carried by the boom can be switched to high precision mode; At the same time, trigger the sound and light prompt, the HMI interface of the control room pops up a yellow warning box, if there is an abnormal parameter, the abnormal parameter is displayed, and the working condition parameters are recorded in the blockchain log in real time, the abnormal parameters are, such as the cargo weight exceeds the rated load, the contact pressure when the hook is locked exceeds , etc;

[0098] 3) According to the intervention level signal to trigger the emergency operation mode and manual confirmation process, through the emergency operation mode to improve the operation safety, and introduce human intervention for troubleshooting, the operator manually confirms whether to continue operation, specifically, including the following contents;

[0099] a. The hook automatically switches to the double safety mode, the main lock remains closed, and the standby mechanical lock is activated; the power system of the lifting equipment automatically runs at reduced power, and the moving speed is limited to 30% of the normal value; b. The multi-dimensional sensing data diagnosis report is automatically pushed to the lifting control system, including the three-dimensional deviation vector diagram of the hook, the contact force change curve of the hook in the last 30 seconds, and the electromagnetic lock current spectrum analysis result; c. The operator completes double verification, such as biometric fingerprint + face recognition to confirm the identity, manually selects the key risk points on the three-dimensional simulation interface for inspection, and if the fault point is locked, it enters the fault handling process, and if it is confirmed to be safe, it continuously presses the emergency operation lever to release the lock and restore normal operation.

[0100] 4) Receiving an emergency level signal directly executes emergency braking, the lifting system autonomously executes deep braking, uses electromagnetic brake + hydraulic damping joint action, and cuts off the power source, forcibly opens the main circuit contactor, and the UPS power source takes over the power supply of the key sensors; the wireless emergency communication channel is started, bypassing the conventional network to directly connect to the control center; the mechanical locking device is activated, the hydraulic locking device completes the full locking of the lifting arm, the emergency counterweight is automatically thrown, and the lifting arm moment is quickly reduced; the alarm signal is sent to the control center, the on-site rotating red warning light and pulse alarm sound are automatically started, and the port broadcast system automatically broadcasts bilingual warnings.

[0101] In an exemplary embodiment of the present disclosure, the automatic hooking and unhooking method further comprises: step S6, coefficient model upgrading and threshold value iterative optimization: dynamic threshold calibration and algorithm model upgrading are performed according to traceable and reliable data records, forming an automatic closed-loop system integrating precise identification, intelligent decision-making, safety control, and self-iteration, significantly improving the safety and efficiency of hooking and unhooking operations, and empowering the intelligent upgrading of ports; the intelligent analysis module sets an optimization trigger mechanism, which includes proactive optimization, periodic optimization, and event-driven optimization; wherein,

[0102] The proactive optimization includes retraining and upgrading of the coefficient model based on incremental data, and by introducing transfer learning, the coefficient model verified by other ports for effective anti-interference is migrated to the local coefficient model;

[0103] The periodic optimization is automatically started according to a preset operation period or a preset operation amount, such as 1000 times of cumulative effective operation or 30 days, whichever comes first, to automatically start the periodic optimization process;

[0104] Periodic optimization includes setting the confidence interval dynamic adjustment threshold of the stored data. If the set confidence interval is 95%, the distribution of the hook coefficient H and the pressure coefficient P of the successful unhooking in the trusted data is extracted, and the reference threshold value that meets the 95% confidence interval is calculated as the new threshold value. For example, if the original hook coefficient threshold H is 0.3, through statistical analysis, 95% of the successful unhooking cases in actual safe operation have a hook coefficient ≤0.35, and the new threshold value can be relaxed to 0.35;

[0105] Periodic optimization also includes mining potential associated factors from stored data and introducing a composite influence index upgrade coefficient model. For example, according to the coupling relationship between wind speed and swing angle mined from stored data, when ≤ , the wind has a sustained boosting effect on the swing angle, and the linear coupling relationship between the two is represented by , , which can ensure that the wind influence is always in the same direction as the swing amplitude, and introduces the conversion efficiency k1 of wind energy-mechanical energy, thereby increasing the composite influence index upgrade swing disturbance term of wind and swing angle:

[0106]

[0107] Event-driven optimization triggers special optimization thresholds according to the number of misjudgment types or the lifting of special goods:

[0108] 1) Turning optimization is performed on the misjudgment types that reach the number of times, the loss is quantified, and the threshold is rebalanced by the ROC curve, which includes the following contents:

[0109] For example, the number of emergency brake mis-touches is W, the efficiency loss weight of misjudgment is 1, the number of correct emergency braking is S, the number of accidents caused by emergency brake false negatives is L, the accident loss weight is 10, and the normal number of emergency braking is Z. The loss function of emergency braking is established as:

[0110] ;

[0111] The total loss of each joint threshold t is calculated (t), and the ROC curve of the total loss is drawn with the horizontal axis X=W / (W+S) and the vertical axis Y=L / (L+H). The optimal threshold value that minimizes the total loss is calculated:

[0112]

[0113] 2) For the lifting of special goods that are overweight, i.e., when M>1.2 When the load compensation mechanism is triggered according to the confirmation, the short-term overload operation is authorized, the original initial threshold P of the pressure coefficient is compensated, and the overweight cargo pressure coefficient threshold P' under the overload operation is generated:

[0114]

[0115] In an exemplary embodiment of the present disclosure, the automatic hooking and unhooking method also considers other work environment disturbances other than the environmental wind speed, and the environmental disturbances reduce the sensing accuracy. The disturbance term adaptation is achieved by increasing the environmental adaptive weight, which includes the following contents:

[0116] 1) Detecting rainfall and rainfall intensity When the rain interferes with the visual sensor group, the weight coefficient of the position deviation disturbance term is increased to generate the adaptive weight coefficient ':

[0117]

[0118] 2) Detecting foggy weather, The minimum visibility is 100 m, The regular visibility is 200 m, and when the visibility V < 100 m, the weight coefficient of the wind resistance disturbance term is increased to generate the adaptive weight coefficient to compensate for the decrease in visual positioning accuracy ':

[0119]

[0120] It can be understood that the hooking coefficient model after the adaptive weight adjustment still satisfies the allocation requirement that the total weight coefficient is 1, and the weight coefficient that is not adjusted adaptively is reduced or the adaptive proportion is reduced.

[0121] The present embodiment also provides a cargo automatic hooking and unhooking system, as shown in Figure 2 Based on the above hooking and unhooking method, the intelligent hooking and unhooking is realized, wherein the hooking mentioned in the present application refers to automatic unlocking hooking, and the double lock catch is provided, the main lock catch is an electromagnetic lock, and the standby lock catch is a mechanical lock. The automatic hooking and unhooking system is connected with the hoisting general control system, and feedback monitoring data, bidirectional feedback decision data and control instructions are provided, which includes a sensing system, an intelligent analysis module, a hierarchical execution control module and a storage module; wherein,

[0122] The perception system is used for monitoring and intelligent control of the unhooking process, and the data source collection includes a visual sensor group, an inclination sensor, a gyroscope IMU, a digital anemograph, a six-dimensional force sensor, a pressure sensor group, a current sensor, and a laser scanner; the visual sensor group includes a binocular camera and a ToF depth sensor, which are used for real-time scanning and jointly determining the spatial position coordinate data of the hook and the sling ring; the inclination sensor collects the swing angle of the sling; the gyroscope IMU determines the swing amplitude of the cargo; the digital anemograph collects real-time wind speed data; the six-dimensional force sensor collects the contact pressure data of the cargo hook when it engages with the sling ring; the pressure sensor group collects the weight data of the cargo; the current sensor is connected to the electromagnetic lock control circuit and collects the current data of the electromagnetic lock; and the laser scanner captures the opening angle of the hook during the unlocking and locking processes of the hook, feeds back the "unloading completed" data to the hoisting control system when the unlocking is completed, and feeds back the "hooking completed" data to the hoisting control system when the locking is completed.

[0123] The intelligent analysis module includes a disturbance term calculation unit, a coefficient fusion calculation unit, a decision unit, and an optimization unit; the disturbance term calculation unit calculates initial disturbance terms or adaptive disturbance terms in a disturbed environment according to the data collected by the perception system; the coefficient fusion calculation unit calculates the hook coefficient and the pressure coefficient according to the weight coefficient optimized by the optimization unit and the coefficient model; the decision unit presets a threshold value and generates a corresponding control instruction according to the calculation result of the coefficient fusion calculation unit and sends it to the hierarchical execution control module; and the optimization unit triggers the optimization of the weight coefficient or the coefficient model according to the real-time data, the historical data, and the incremental data in the storage module by the optimization trigger mechanism and feeds back to the coefficient fusion calculation unit.

[0124] The hierarchical execution control module includes a hierarchical control end and a hierarchical execution end; the hierarchical control end receives the decision instruction of the decision unit, generates a corresponding control signal, and sends it to each node of the hierarchical execution end and the hoisting control system; the hierarchical execution end includes an electromagnetic lock, a mechanical lock, a perception system, and a manual troubleshooting guide system; the hierarchical control end triggers the electromagnetic lock to unlock according to the unhooking instruction of the decision unit; triggers the perception system to intensify monitoring according to the early warning level instruction, and synchronously sends it to the hoisting control system to improve the operation accuracy of the boom and give a pop-up window and an audible and visual warning for abnormal parameters; triggers the emergency operation mode and the manual troubleshooting guide system according to the intervention level instruction to improve the safety of operation in abnormal state, calls the perception data of the key risk points, introduces human intervention, and selects the key risk points for inspection and confirmation by the operator to exclude faults and consider the lock to be released; and sends an emergency braking signal to the hoisting control system according to the emergency level instruction, including cutting off the power supply, deep braking of the hoisting system, boom locking, and audible and visual warning.

[0125] In an exemplary embodiment of the present disclosure, in order to continuously optimize the algorithm model and threshold parameters with the notarized data, the storage module stores the operation log through the blockchain to ensure that the operation log is not tamperable, the decision-making process is fully traceable, the historical data source is reliable, the port compliance audit requirements are met, the system is self-adaptive and optimized to improve the success rate of hooking and stabilize; each round of operation generates a log-structured data block, and the blockchain log structure includes:

[0126] Operation fingerprint: timestamp, spreader ID, cargo type, environment label, etc.

[0127] Raw data of the perception system: visual positioning deviation, hook pressure value, electromagnetic lock current waveform segment, etc.

[0128] Decision-making process data: double coefficient calculation value, threshold determination result, execution action, etc.

[0129] Artificial correction record: operator's identity information, parameter adjustment content and reason note, etc.

[0130] In addition, the storage module adopts a consortium chain architecture, taking the terminal management party, equipment manufacturer and regulatory agency as consensus nodes, to provide a complete evidence chain for accident traceability and solve the problem of responsibility definition in traditional black box systems; high-quality scene data is accumulated to avoid dirty data pollution caused by sensor noise or communication packet loss.

[0131] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments of the present application can be realized by means of software or software combined with necessary general hardware platforms, and of course can also be realized by hardware functions. Based on this understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium, including a plurality of instructions for causing a computer device, such as but not limited to a personal computer, a server, or a network device, to execute all or part of the steps of the method described in any embodiment of the present application.

[0132] Although the embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, replacements and variations can be made to the embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A method for automatically unhooking goods based on machine vision technology, characterized in that: The following steps are involved: S1. Data Collection: The three-dimensional coordinates of the hook and the eye are collected using visual technology, and hook-related data and environmental data are collected using multi-source perception. The hook-related data includes the hook's inclination angle, pressure, load, and device status, and the environmental data includes ambient wind speed. S2. Data preprocessing: establishing a dynamic three-dimensional coordinate system based on the three-dimensional position coordinates of the lifting ring, calculating the position deviation of the hook, and establishing at least a position deviation disturbance term, a swing disturbance term, a wind resistance disturbance term, a force balance disturbance term, a load matching disturbance term, and an equipment health disturbance term based on the hook-related data and the environment-related data; S3. Constructing a coefficient model integrating related disturbance terms: integrating at least the position deviation disturbance term, the swing disturbance term, and the wind resistance disturbance term to establish a hook coefficient model and calculate the hook coefficient; integrating at least the force balance disturbance term, the load matching disturbance term, and the equipment health disturbance term to establish a pressure coefficient model and calculate the pressure coefficient; S4. Generate instructions: A dual-coefficient joint decision-making mechanism is used to make decoupling decisions. Hook coefficient thresholds, pressure coefficient thresholds, and dual-coefficient joint thresholds are set at each level. If the hook coefficient and the pressure coefficient meet the decoupling conditions, a decoupling instruction is issued; if they do not meet the decoupling conditions, a warning signal is issued. S5. Trigger different processing tasks according to the hook removal instruction or the early warning signal.

2. The automatic cargo hook removal method based on machine vision technology according to claim 1 is characterized in that: It also includes the upgrade of the coefficient model and iterative optimization of thresholds, and the setting of optimization trigger mechanisms, including active optimization, periodic optimization, and event-driven optimization; The active optimization fine-tunes the coefficient model based on incremental data and introduces transfer learning to migrate the off-site coefficient model that has been verified to be effective in resisting interference at other terminals to the local terminal for optimization; The periodic optimization includes: setting a dynamic adjustment threshold of the confidence interval of credible data according to a preset operation cycle or a preset operation volume, and mining potential correlation factors based on the credible data, introducing composite influence indicators to upgrade the coefficient model; the event-driven optimization triggers special optimization according to the number of misjudgment types or special cargo lifting.

3. The automatic cargo hook removal method based on machine vision technology according to claim 2 is characterized in that: The hook coefficient model is: The linkage coefficient ; Position deviation coefficient ; Swing coefficient ; Drag compensation coefficient ; in, 、 and are the weight coefficients of the position deviation disturbance term, the swing disturbance term and the wind resistance disturbance term, respectively, and + + =1; is the three-dimensional position deviation between the hook and the ring, that is, ( , , ), using the Euclidean norm The modulus length that measures the three-dimensional position deviation, is the maximum allowable deviation modulus; is the swing angle of the hook, k is the angle suppression factor; is the real-time wind speed, is the critical wind speed.

4. The automatic cargo hook removal method based on machine vision technology according to claim 3 is characterized in that: The pressure coefficient model is: The pressure coefficient ; Force balance coefficient ; Load matching coefficient ; Equipment health coefficient ; in, 、 and are the weight coefficients of the force balance disturbance term, the load matching disturbance term, and the equipment health disturbance term, respectively, and + + =1; is the real-time contact pressure of the hook, For optimal contact pressure, is the maximum allowable contact pressure, is the minimum permissible contact pressure; is the weight of the cargo, is the rated load of the hook; is the actual working current of the hook electromagnetic lock, is the nominal current of the hook electromagnetic lock.

5. The automatic cargo hook removal method based on machine vision technology according to claim 2 is characterized in that: When other working environment interferences other than the ambient wind speed occur, the environmental adaptation weight is increased, including the following: 1) Detect rainfall and rainfall intensity When the weight coefficient of the position deviation disturbance term is increased, an adaptive weight coefficient is generated. ': ； 2) Haze weather is detected and visibility V < , increase the weight coefficient of the wind resistance disturbance term to generate an adaptive weight coefficient ': ； in, For minimum visibility, Normal visibility.

6. The automatic cargo hook removal method based on machine vision technology according to claim 4 is characterized in that: The unhooking threshold h of the hook coefficient and the unhooking threshold p of the pressure coefficient are preset. When H≤h and P≤p, the unhooking condition is met and the unhooking instruction is sent. For situations where the unhooking instruction is not met, at least three warning levels are set to generate different decision instructions. The unhooking urgency thresholds h1 and h2 of the hook coefficient, the unhooking urgency thresholds p1 and p2 of the pressure coefficient, and the dual-coefficient joint thresholds t and T are preset. The warning level is set as: Warning level: , and H+P≤t; or, , and H+P≤t; triggering state monitoring enhancement to improve operation accuracy; Intervention level: ;or ; or H+P ;Trigger emergency operation mode and manual confirmation process; Emergency level: H> ; or P> ; or H+P> ; trigger emergency braking, cut power and send alarm.

7. The method for automatically unhooking cargo based on machine vision technology according to claim 2, characterized in that: For the misjudgment type that reaches a certain number of times, the loss is quantified, and the threshold is rebalanced through the ROC curve. The optimization balance process includes the following: Define the number of misjudgments of a certain type of dangerous event as W, and the efficiency loss weight of misjudgment as , the number of correct identification and triggering is S, the number of missed reports is L, and the accident loss weight of missed reports is , the normal number of times without false triggering is H, and the loss function is established: ； Calculate the total loss at each threshold t (t), with the horizontal axis X=W / (W+S) and the vertical axis Y=L / (L+H), create an ROC curve and calculate the optimal threshold that minimizes the total loss : 。 8. The method for automatically unhooking cargo based on machine vision technology according to claim 4, characterized in that: For overweight special cargo, i.e. M> hour, >1, the load compensation mechanism is triggered, and the overweight cargo pressure coefficient threshold P' is used to allow short-term overload operations: 。 9. An automatic cargo hook removal system, based on the automatic cargo hook removal method based on machine vision technology according to any one of claims 1 to 8, characterized in that: The hook is an automatic unlocking hook with a double lock, wherein the main lock is an electromagnetic lock and the backup lock is a mechanical lock. The automatic hook removal system is connected to the lifting control system, which includes a perception system, an intelligent analysis module, a hierarchical execution control module and a storage module; wherein, The perception system includes a visual sensor group, an inclination sensor, a gyroscope IMU, a digital anemometer, a six-dimensional force sensor, a pressure sensor group, a current sensor and a laser scanner; the visual sensor group includes a binocular camera and a ToF depth sensor arranged at the front end of the gantry crane beam or boom, for real-time scanning of the three-dimensional position data of the hook and the lifting ring; the inclination sensor and the gyroscope IMU are arranged at the connection between the hook and the spreader, for collecting and monitoring the swing angle and amplitude of the spreader; the digital anemometer is installed at the top of the boom, for collecting real-time wind speed data; the six-dimensional force sensor is installed at the connection between the hook and the lifting ring, for collecting the contact pressure data of the hook; the pressure sensor group is built into the spreader, for collecting cargo weight data; the current sensor is connected to the electromagnetic lock control circuit, for collecting the current data of the electromagnetic lock; the laser scanner is arranged on the hook, for capturing the unlocking and locking process of the hook; The intelligent analysis module includes a disturbance term calculation unit, a coefficient fusion calculation unit, a decision unit, and an optimization unit. The disturbance term calculation unit calculates various disturbance terms based on the data collected by the perception system. The coefficient fusion calculation unit calculates the hook coefficient and the pressure coefficient based on the preset weight coefficient and coefficient model or the weight coefficient optimized by the optimization unit. The decision unit analyzes the result of the coefficient fusion calculation unit according to a preset threshold and generates corresponding control instructions. The optimization unit triggers the optimization of the weight coefficient or the coefficient model according to the optimization trigger mechanism based on the real-time data, historical data, and incremental data in the storage module. The hierarchical execution control module includes a hierarchical control end and a hierarchical execution end. The hierarchical execution end includes the electromagnetic lock, the mechanical lock, the perception system and the manual inspection and guidance system. The hierarchical control end triggers the electromagnetic lock to unlock according to the unhooking instruction of the decision unit; triggers the perception system to strengthen monitoring according to the early warning level instruction, and sends a signal to improve operation accuracy to the lifting control system; triggers the manual inspection and guidance system according to the intervention level instruction, activates the backup lock and sends an emergency operation mode signal to the lifting control system to improve the safety of abnormal operations and introduce human intervention; sends an emergency braking signal to the lifting control system according to the emergency level instruction.

10. The automatic cargo hook removal system according to claim 9, characterized in that: The storage module uses a blockchain log structure to store trusted data and adopts a consortium chain architecture, with terminal managers, equipment manufacturers, and regulatory agencies as consensus nodes. The blockchain log structure includes: Operation fingerprint: including timestamp, spreader ID, cargo type, and environmental tag; Raw data from the perception system: including visual positioning deviation, hook pressure value, and electromagnetic lock current waveform fragment; Decision-making process data: including double coefficient calculation values, threshold determination results, and execution actions; Manual correction records: include parameter adjustment contents and reasons when the operator intervenes.

Images