Bulk material grab bucket control system based on machine vision

Abstract

The invention discloses a bulk cargo grab bucket control system based on machine vision, relates to the technical field of automatic control, and solves the technical problems that in the prior art, a bulk cargo containing container is likely to make contact with a grab bucket to cause damage to the grab bucket, and the bulk cargo grabbing efficiency of the grab bucket is low due to the fact that the same control mode is adopted for different materials. Obtaining a material label through the material identification model; generating a grabbable area according to the material data and the grab bucket parameters; obtaining a target coordinate through a laser radar technology; obtaining an opening angle and a closing angle in the grab bucket parameters according to the material data; grab bucket adjustment parameters are generated according to the grab bucket coordinates and the target coordinates; the grab bucket adjusting parameters are assigned to the corresponding grab bucket parameters for control, the control parameters of the grab bucket are adjusted in real time along with changes of materials and changes of the grabbing environment, the grabbing area is planned for grabbing, contact between the grab bucket and the containing container is reduced, the material grabbing efficiency is improved, and the damage rate of the containing container is reduced.

Description

Technical Field

[0001] The present application belongs to the field of automation control technology, and specifically is a bulk material grab control system based on machine vision. Background Art

[0002] Grab bucket is a special tool for cranes to grab dry bulk cargoes. It is composed of two or more bucket-shaped jaws that can be opened and closed to form a storage space. When loading, the jaws are closed in the material pile and the materials are grabbed into the storage space; when unloading, the jaws are opened in a suspended state above the material pile, and the materials are scattered on the material pile. The opening and closing of the jaws are generally controlled by the steel wire rope of the crane hoisting mechanism. Grabs can be divided according to various factors such as purpose, working principle, shape, and driving method.

[0003] The existing bulk material grab control system mainly relies on manual operation or simple sensors for positioning and control. There is a problem that the bulk material container is easily brought into contact with the grab, causing damage to the grab. In addition, adopting the same control method for different materials will make the grab's efficiency in grabbing bulk materials low. Therefore, the bulk material grab control system still needs further improvement. Summary of the invention

[0004] The present application aims to solve at least one of the technical problems existing in the prior art; to this end, the present application proposes a bulk material grab control system based on machine vision, which is used to solve the technical problems in the prior art that the bulk material container is easily brought into contact with the grab, causing damage to the grab, and that the same control method is used for different materials, resulting in low efficiency in grabbing bulk materials.

[0005] To achieve the above-mentioned purpose, the first aspect of the present application provides a bulk material grab control system based on machine vision, including: a data acquisition module, a data analysis module, a control module, an early warning module and a database;

[0006] The data acquisition module is used to acquire material data, grab parameters and environmental data through data acquisition equipment;

[0007] The data analysis module: inputs the material image in the material data into the material recognition model to obtain the material label, and the material recognition model is constructed by the artificial intelligence model; generates the grabbable area according to the material data and the grab parameters; obtains the target coordinates in the grabbable area in real time by the laser radar technology; obtains the opening angle and closing angle in the grab parameters according to the material data; obtains the grab coordinates in real time, and generates the grab adjustment parameters according to the grab coordinates and the target coordinates; generates an alarm signal according to the grab data;

[0008] The control module: assigns the grab adjustment parameters to the corresponding grab parameters for control;

[0009] The early warning module: makes prompts according to the alarm signal and contacts the management personnel.

[0010] The present application obtains the category label and characteristics of the grasped material through the above steps, and adaptively adjusts the corresponding parameters for it to meet the grasping of materials in various situations; at the same time, when grasping materials, the grasping area is limited to avoid contact between the grab bucket and the container containing the materials, reduce the occurrence of container damage, and improve the material grasping efficiency.

[0011] Furthermore, the material recognition model is constructed through an artificial intelligence model, including:

[0012] Obtain several historical material images and their corresponding material labels and material sizes;

[0013] Divide a number of historical material images and their corresponding material labels and material sizes into training data, verification data, and test data; perform data preprocessing on the training data, verification data, and test data to obtain a training set, a verification set, and a test set;

[0014] Select an AI model as the base model;

[0015] Train the basic model with the training set, and adjust the learning rate and hyperparameters on the validation set to obtain the pre-trained model;

[0016] By verifying the pre-trained model on the test set, we finally obtained a material recognition model whose input is material image and output is material label and material size.

[0017] Furthermore, generating the grabbable area according to the material data and grab bucket parameters includes:

[0018] Obtain the material area in the material data, the grab bucket size in the grab bucket parameters, and the center position of the material area; the material area includes a length WL and a width WK; the grab bucket size includes a length ZL and a width ZK;

[0019] The length KZL of the graspable area is calculated by the formula KZL = WL-2×ZL;

[0020] The width KZK of the graspable area is calculated by the formula KZK=WK-2×ZK;

[0021] The gripping area is constructed based on the center position of the material area, the length KZL and the width KZK.

[0022] Furthermore, the real-time acquisition of the target coordinates within the graspable area by using the laser radar technology includes the following steps:

[0023] Step 1: Start the laser radar device to scan the graspable area and collect 3D point cloud data in real time;

[0024] Step 2: Preprocessing the three-dimensional point cloud data to obtain preprocessed three-dimensional point cloud data;

[0025] Step 3: Select the position with the largest vertical coordinate value in the 3D point cloud data as the target coordinate.

[0026] This application uses lidar technology to visualize the material distribution status within the graspable area, and selects the material position corresponding to the largest vertical coordinate value in the three-dimensional point cloud data within the graspable area as the grasping position in turn, ensuring that the volume of material grasped each time can meet the grasping requirements and improving the efficiency of material grasping.

[0027] Further, obtaining the opening angle and closing angle of the grab bucket parameters according to the material data includes:

[0028] Extract material labels and material dimensions;

[0029] Find the corresponding bulk density and surface condition in the material properties table according to the material label;

[0030] Integrate material labels and their corresponding material dimensions, bulk density and surface conditions into comprehensive material data;

[0031] The comprehensive material data is input into the angle prediction model to obtain the opening angle and closing angle of the material. The angle prediction model is constructed through an artificial intelligence model.

[0032] Furthermore, the angle prediction model is constructed by an artificial intelligence model, including:

[0033] Obtain several historical material labels, material dimensions, bulk density and surface conditions and their corresponding opening and closing angles;

[0034] Integrate several historical material labels, material dimensions, bulk density and surface conditions into several historical material comprehensive data;

[0035] Divide a number of historical material comprehensive data and their corresponding opening angles and closing angles into training data, verification data and test data; perform data preprocessing on the training data, verification data and test data to obtain a training set, a verification set and a test set;

[0036] Select an AI model as the base model;

[0037] Train the basic model with the training set, and adjust the learning rate and hyperparameters on the validation set to obtain the pre-trained model;

[0038] By verifying the pre-trained model on the test set, we finally obtained an angle prediction model whose input is comprehensive material data and output is the opening angle and closing angle.

[0039] Further, generating the grab adjustment parameters according to the grab coordinates and the target coordinates includes:

[0040] Obtaining the target coordinates and the grab volume;

[0041] Generating the grab adjustment height of the grab according to the target coordinates and the grab volume;

[0042] Obtaining the grab coordinates and the corresponding wind force magnitude in real time;

[0043] Calculating the grab movement adjustment speed according to the grab coordinates and the corresponding wind force magnitude.

[0044] Further, generating the grab adjustment height of the grab according to the target coordinates and the grab volume includes:

[0045] Obtaining the grab coordinates, the target coordinates, the grab volume ZDV, and the grab height ZDG;

[0046] When the vertical coordinate in the target coordinates is less than the grab height, setting the grab adjustment height to the value corresponding to the grab height;

[0047] Otherwise, constructing the grab point cloud data according to the opening angle of the grab;

[0048] Dividing the grab point cloud data according to the standard unit height BDG, and calculating the volume TVi of each standard unit height through the voxelization method; where, i = 1, 2,..., N; N represents the total number after division, and i = 1 is the number corresponding to the top in the grab point cloud data;

[0049] Obtaining the i that satisfies the following conditions; that is, ∑(TVi) < ZDV and ∑(TVi + 1) ≥ ZDV; the ∑ summation range is [1, N];

[0050] When the volume of the grab point cloud data is less than ZDV, setting the grab adjustment height to the value corresponding to the grab height; otherwise, calculating the grab adjustment height through the formula ZTG = BDG × (N - (i + 1)) + ZDG.

[0051] In this application, by constructing the grab point cloud data according to the opening angle of the grab in the above steps, that is, the area covered by the vertical downward movement of the grab opening angle is the grab area, considering that the volume of the material grabbed each time meets the requirements of the grab volume, avoiding the problem of low grab efficiency caused by the volume of the material grabbed each time being much smaller than the grab volume, and improving the grab efficiency.

[0052] Further, constructing the grab point cloud data according to the opening angle of the grab includes:

[0053] Step 1: Take the range of grab opening angle as the boundary and the vertical coordinate of the target coordinate as the height to construct a corresponding three-dimensional geometric model;

[0054] Step 2: Align and register the three-dimensional point cloud data corresponding to the material with the three-dimensional geometric model through the ICP algorithm;

[0055] Step 3: Using a 3D point cloud processing algorithm, the 3D point cloud data is cut according to the boundary of the solid geometric model to obtain captured point cloud data.

[0056] Further, the calculation of the grab bucket movement adjustment speed according to the grab bucket coordinates and the corresponding wind force magnitude includes:

[0057] Obtain the grab bucket coordinates and its corresponding wind force FD and grab adjustment height ZTG and its corresponding wind force ZTFD in real time;

[0058] When the vertical coordinate ZZ in the grab bucket coordinates is greater than the sum of the grab adjustment height and the adjustment threshold, the formula Calculate the grab moving adjustment speed ZYTS;

[0059] Otherwise, the grab movement adjustment speed ZYTS is calculated by the formula ZYTS=β×BS×exp(-(α1×ZTG / DG+α2×ZTFD / DFD)); wherein α1 and α2 are weight coefficients, α1 and α2∈(0,1); β is the proportional coefficient, β∈(0,1); is the rounding symbol, DG is the unit height, DFD is the unit wind force, and BS is the standard speed.

[0060] Further, generating an alarm signal according to the grab bucket data includes:

[0061] Get the grab bucket coordinates and the two-dimensional plane area of ​​the grabbable area;

[0062] Map the horizontal and vertical coordinates in the grab bucket coordinates to the two-dimensional plane to obtain the two-dimensional coordinates;

[0063] When the two-dimensional coordinate exceeds the two-dimensional plane area of ​​the graspable area, a grab bucket deviation alarm signal is generated; otherwise, no operation is performed.

[0064] Compared with the prior art, the beneficial effects of this application are:

[0065] 1. The present application obtains a material label by inputting a material image in material data into a material identification model; generates a graspable area according to material data and grab parameters; obtains target coordinates in the graspable area in real time through lidar technology; obtains an opening angle and a closing angle in grab parameters according to material data; obtains grab coordinates in real time, and generates grab adjustment parameters according to grab coordinates and target coordinates; assigns the grab adjustment parameters to their corresponding grab parameters for control, adjusts the grab control parameters in real time as the material changes and the grabbing environment changes, and plans the graspable area for grabbing, which reduces the contact between the grab and the container, improves the material grabbing efficiency, and reduces the damage rate of the container.

[0066] 2. This application generates a grabbable area through the material area in the material data, the grab size in the grab parameters, and the center position of the material area, and grabs the material within the grabbable area, thereby reducing the contact between the grab and other objects and avoiding damage to other objects.

[0067] 3. This application constructs grabbing point cloud data according to the opening angle of the grab bucket. That is, the area that can be covered by the vertical downward movement of the grab bucket opening angle is the grabbing area. The volume of materials grabbed each time is considered to meet the requirements of the grab bucket volume, thereby avoiding the problem of low grabbing efficiency caused by the volume of materials grabbed each time being far smaller than the grab bucket volume, thereby improving the grabbing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0069] Figure 1 This is a schematic diagram of the principle of the bulk material grab control system based on machine vision in this application;

[0070] Figure 2 This is a flow chart of the bulk material grab control method based on machine vision in this application. DETAILED DESCRIPTION

[0071] The technical solution of the present application will be described clearly and completely in conjunction with the embodiments below. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0072] See also Figure 1-Figure 2, the first embodiment of the present application provides a bulk material grab control system based on machine vision, including: a data acquisition module, a data analysis module, a control module, an early warning module and a database;

[0073] Data acquisition module: obtain material data, grab parameters and environmental data through data acquisition equipment; data acquisition equipment includes high-precision cameras, various sensors, etc.;

[0074] Data analysis module: Input the material image in the material data into the material recognition model to obtain the material label. The material recognition model is constructed through the artificial intelligence model; generate the graspable area according to the material data and the grab parameters, and the graspable area will not touch other objects; obtain the target coordinates in the graspable area in real time through the laser radar technology, and the target coordinates are the material targets to be grasped; obtain the opening angle and closing angle in the grab parameters according to the material data; obtain the grab coordinates in real time, and generate the grab adjustment parameters according to the grab coordinates and the target coordinates. The grab adjustment parameters are the adjustment values ​​of the grab parameters; generate an alarm signal according to the grab data;

[0075] Control module: assign grab adjustment parameters to their corresponding grab parameters for control;

[0076] Early warning module: Prompts are given based on alarm signals and management personnel are contacted; alarm signals include grab bucket deviation alarm signals, etc.

[0077] The material recognition model in this embodiment is constructed by an artificial intelligence model, including:

[0078] Obtain several historical material images and their corresponding material labels and material sizes; material labels include sand, gravel, etc.;

[0079] Several historical material images and their corresponding material labels and material sizes are divided into training data, verification data and test data; data preprocessing is performed on the training data, verification data and test data to obtain a training set, a verification set and a test set; the ratio between the training set, the test set and the verification set is 7:2:1;

[0080] Select an artificial intelligence model as the basic model; the artificial intelligence model includes a convolutional neural network model, etc.;

[0081] Train the basic model with the training set, and adjust the learning rate and hyperparameters on the validation set to obtain the pre-trained model;

[0082] By verifying the pre-trained model on the test set, we finally obtained a material recognition model whose input is material image and output is material label and material size.

[0083] In this embodiment, the grabbing area is generated according to the material data and the grab bucket parameters, including:

[0084] Get the material area in the material data, the grab bucket size in the grab bucket parameters, and the center position of the material area; the material area includes the length WL and the width WK; the grab bucket size includes the length ZL and the width ZK; the center position is the center point in the two-dimensional plane of the grab bucket area;

[0085] The length KZL of the graspable area is calculated by the formula KZL = WL-2×ZL;

[0086] The width KZK of the grabbing area is calculated by the formula KZK = WK-2×ZK; once the length and width of the material area are fixed, the length and width of the grabbing area decrease as the size of the grab increases;

[0087] The gripping area is constructed based on the center position of the material area, the length KZL and the width KZK.

[0088] Through the above steps, this embodiment divides the material area into a grabbable area with the edge that the grab bucket can touch as the boundary, and grabs the material within the grabbable area, which can avoid contact between the grab bucket and the material container and reduce the possibility of damage to the container.

[0089] In this embodiment, the laser radar technology is used to obtain the coordinates of the target within the graspable area in real time, including the following steps:

[0090] Step 1: Start the laser radar device to scan the graspable area and collect 3D point cloud data in real time;

[0091] Step 2: Preprocessing the three-dimensional point cloud data to obtain preprocessed three-dimensional point cloud data;

[0092] Step 3: Select the position with the largest vertical coordinate value in the 3D point cloud data as the target coordinate, that is, the position of the material grasped each time is the position corresponding to the highest point of the material in the graspable area.

[0093] In this embodiment, the opening angle and closing angle of the grab bucket parameters are obtained according to the material data, including:

[0094] Extract material labels and material dimensions;

[0095] Find the corresponding bulk density and surface condition in the material property table according to the material label; the material property table is constructed by experts based on the different properties of different materials;

[0096] Integrate material labels and their corresponding material dimensions, bulk density and surface conditions into comprehensive material data;

[0097] The comprehensive material data is input into the angle prediction model to obtain the opening angle and closing angle of the material. The angle prediction model is constructed through an artificial intelligence model.

[0098] The angle prediction model in this embodiment is constructed by an artificial intelligence model, including:

[0099] Obtain several historical material labels, material dimensions, bulk density and surface conditions and their corresponding opening and closing angles;

[0100] Integrate several historical material labels, material dimensions, bulk density and surface conditions into several historical material comprehensive data;

[0101] The comprehensive data of several historical materials and their corresponding opening angles and closing angles are divided into training data, verification data and test data; the training data, verification data and test data are preprocessed to obtain the training set, verification set and test set; the ratio between the training set, the test set and the verification set is 7:2:1;

[0102] Select an artificial intelligence model as the basic model; the artificial intelligence model includes a convolutional neural network model, etc.;

[0103] Train the basic model with the training set, and adjust the learning rate and hyperparameters on the validation set to obtain the pre-trained model;

[0104] By verifying the pre-trained model on the test set, we finally obtained an angle prediction model whose input is comprehensive material data and output is the opening angle and closing angle.

[0105] In this embodiment, the grab adjustment parameters are generated according to the grab coordinates and the target coordinates, including:

[0106] Obtain the target coordinates and grab bucket capacity; the grab bucket capacity is the maximum volume that the grab bucket can hold;

[0107] Generate grab adjustment height of grab bucket according to target coordinates and grab bucket volume;

[0108] Obtain grab bucket coordinates and corresponding wind force in real time;

[0109] The grab bucket movement adjustment speed is calculated based on the grab bucket coordinates and the corresponding wind force.

[0110] In this embodiment, the grab adjustment height of the grab bucket is generated according to the target coordinates and the grab bucket volume, including:

[0111] Obtain grab bucket coordinates, target coordinates, grab bucket volume ZDV and grab bucket height ZDG; grab bucket height refers to the height of the grab bucket in the closed state;

[0112] When the vertical coordinate in the target coordinates is less than the grab bucket height, set the grab adjustment height to the value corresponding to the grab bucket height; that is, the volume of the material corresponding to the target coordinates within the grabable area is relatively small. If the grab height is set lower, the grab bucket will contact the bottom of the storage container during material grabbing, causing damage to it.

[0113] Otherwise, construct the grab point cloud data according to the opening angle of the grab bucket; the grab point cloud data refers to the point cloud data of the material within the grabable area perpendicular to the opening angle of the grab bucket.

[0114] Divide the grab point cloud data according to the standard unit height BDG, and calculate the volume TVi of each standard unit height through the voxelization method. The value of BDG is set according to experience and is set to 10 cm in this embodiment; where i = 1, 2,..., N; N represents the total number after division, and i = 1 is the number corresponding to the top in the grab point cloud data.

[0115] Obtain i that satisfies the following conditions; that is, ∑(TVi) < ZDV and ∑(TVi + 1) ≥ ZDV; the summation range of ∑ is [1, N]; this condition mainly finds the material height that just exceeds the grab bucket volume.

[0116] When the volume of the grab point cloud data is less than ZDV, set the grab adjustment height to the value corresponding to the grab bucket height; otherwise, calculate the grab adjustment height through the formula ZTG = BDG×(N - (i + 1)) + ZDG; when BDG and ZDG are fixed, ZTG decreases as i increases.

[0117] In this embodiment, by constructing the grab point cloud data according to the opening angle of the grab bucket in the above steps, that is, the area covered by the vertical downward movement of the opening angle of the grab bucket is the grab area, considering that the volume of the material grabbed each time meets the requirements of the grab bucket volume, avoiding the problem of low grab efficiency caused by the volume of the material grabbed each time being much less than the grab bucket volume, and improving the grab efficiency of the material.

[0118] The construction of the grab point cloud data according to the opening angle of the grab bucket in this embodiment includes:

[0119] Step 1: Construct a corresponding three-dimensional geometric model with the range of the opening angle of the grab bucket as the boundary and the vertical coordinate of the target coordinates as the height.

[0120] Step 2: Align and register the three-dimensional point cloud data corresponding to the material with the three-dimensional geometric model through the ICP algorithm.

[0121] Step 3: Use the three-dimensional point cloud processing algorithm to cut the three-dimensional point cloud data according to the boundary of the three-dimensional geometric model to obtain the grab point cloud data.

[0122] In this embodiment, the calculation of the grab bucket movement adjustment speed according to the grab bucket coordinates and the corresponding wind force magnitude includes:

[0123] Obtain the grab bucket coordinates and its corresponding wind force FD and grab adjustment height ZTG and its corresponding wind force ZTFD in real time;

[0124] When the vertical coordinate ZZ in the grab bucket coordinates is greater than the sum of the grab adjustment height and the adjustment threshold, the formula Calculate the grab bucket moving adjustment speed ZYTS; round up the sign This is to prevent the movement speed from changing frequently due to the change of short height. As ZZ and FD increase, the movement speed of the grab bucket needs to be reduced to reduce the occurrence of dangerous situations. For example, when the wind is strong, the movement speed of the grab bucket needs to be reduced. If it moves quickly, some safety hazards may occur.

[0125] Otherwise, the grab movement adjustment speed ZYTS is calculated by the formula ZYTS=β×BS×exp(-(α1×ZTG / DG+α2×ZTFD / DFD)). The setting of β is to reduce the movement speed. When approaching the target coordinates, the movement speed is reduced to help the user adjust the grab state. Therefore, the grab movement adjustment speed will be reduced accordingly; wherein, α1 and α2 are weight coefficients, α1 and α2∈(0,1), and the specific values ​​are set according to experience; β is the proportional coefficient, β∈(0,1), and the specific value is set according to experience; is the rounding symbol, DG is the unit height, DFD is the unit wind force, BS is the standard speed, and the specific values ​​are set according to experience. In this embodiment, DG is set to 1m and DFD is set to 1m / s.

[0126] This embodiment calculates the moving speed of the grab bucket during movement according to its coordinates and the corresponding wind force, and reduces the moving speed when the wind force is strong and the height is high, thereby increasing the safety of the grab bucket. When the grab bucket is about to reach the grabbing height, reducing the speed is conducive to the user controlling the grab bucket for adjustment, thereby achieving effective control of the grab bucket.

[0127] In this embodiment, generating an alarm signal according to grab bucket data includes:

[0128] Get the grab bucket coordinates and the two-dimensional plane area of ​​the grabbable area;

[0129] Map the horizontal and vertical coordinates in the grab bucket coordinates to the two-dimensional plane to obtain the two-dimensional coordinates;

[0130] When the two-dimensional coordinate exceeds the two-dimensional plane area of ​​the graspable area, a grab offset alarm signal is generated; otherwise, no action is taken.

[0131] Part of the data in the above formula is calculated by removing the dimension and taking its numerical value. The formula is a formula closest to the actual situation obtained by software simulation of a large amount of collected data; the preset parameters and preset thresholds in the formula are set by technical personnel in this field according to actual conditions or obtained through simulation of a large amount of data.

[0132] The working principle of the present application is as follows: by acquiring material data, grab parameters and environmental data; inputting the material image in the material data into the material recognition model to obtain the material label; generating a graspable area according to the material data and grab parameters; acquiring the target coordinates in the graspable area in real time through lidar technology; obtaining the opening angle and closing angle in the grab parameters according to the material data; acquiring the grab coordinates in real time, generating the grab adjustment parameters according to the grab coordinates and the target coordinates, and assigning the grab adjustment parameters to the corresponding grab parameters for control; generating an alarm signal according to the grab data, making a prompt according to the alarm signal, and contacting the management personnel, adjusting the control parameters of the grab in real time as the material changes and the grabbing environment changes, and planning the graspable area for grabbing to reduce the contact between the grab and the container, thereby improving the material grabbing efficiency and reducing the damage rate of the container, avoiding the problem in the prior art that the container for bulk materials is easily brought into contact with the grab, causing damage to it, and that adopting the same control method for different materials will make the efficiency of the grab grabbing bulk materials low.

[0133] The above embodiments are only used to illustrate the technical method of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the preferred embodiments, a person of ordinary skill in the art should understand that the technical method of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present application.

Claims

1. Bulk material grab control system based on machine vision, characterized in that: include: Data acquisition module, data analysis module, control module, early warning module and database; The data acquisition module is used to acquire material data, grab parameters and environmental data through data acquisition equipment; The data analysis module: inputs the material image in the material data into the material recognition model to obtain the material label, and the material recognition model is constructed by the artificial intelligence model; generates the grabbable area according to the material data and the grab parameters; obtains the target coordinates in the grabbable area in real time by the laser radar technology; obtains the opening angle and closing angle in the grab parameters according to the material data; obtains the grab coordinates in real time, and generates the grab adjustment parameters according to the grab coordinates and the target coordinates; generates an alarm signal according to the grab data; The control module assigns the grab adjustment parameters to the corresponding grab parameters for control.

2. The bulk material grab control system based on machine vision according to claim 1, characterized in that: The material recognition model is constructed through an artificial intelligence model, including: Obtain several historical material images and their corresponding material labels and material sizes; Divide a number of historical material images and their corresponding material labels and material sizes into training data, verification data, and test data; perform data preprocessing on the training data, verification data, and test data to obtain a training set, a verification set, and a test set; Select an AI model as the base model; Train the basic model with the training set, and adjust the learning rate and hyperparameters on the validation set to obtain the pre-trained model; By verifying the pre-trained model on the test set, we finally obtained a material recognition model whose input is material image and output is material label and material size.

3. The bulk material grab control system based on machine vision according to claim 1, characterized in that: The generating of the grabbing area according to the material data and grabbing bucket parameters includes: Obtain the material area in the material data, the grab bucket size in the grab bucket parameters, and the center position of the material area; the material area includes a length WL and a width WK; the grab bucket size includes a length ZL and a width ZK; The length KZL of the graspable area is calculated by the formula KZL = WL-2×ZL; The width KZK of the graspable area is calculated by the formula KZK=WK-2×ZK; The gripping area is constructed based on the center position of the material area, the length KZL and the width KZK.

4. The bulk material grab control system based on machine vision according to claim 1, characterized in that: The real-time acquisition of the target coordinates within the graspable area by using the laser radar technology includes the following steps: Step 1: Start the laser radar device to scan the graspable area and collect 3D point cloud data in real time; Step 2: Preprocessing the three-dimensional point cloud data to obtain preprocessed three-dimensional point cloud data; Step 3: Select the position with the largest vertical coordinate value in the 3D point cloud data as the target coordinate.

5. The bulk material grab control system based on machine vision according to claim 1, characterized in that: The method of obtaining the opening angle and closing angle of the grab bucket parameters according to the material data includes: Extract material labels and material dimensions; Find the corresponding bulk density and surface condition in the material properties table according to the material label; Integrate material labels and their corresponding material dimensions, bulk density and surface conditions into comprehensive material data; The comprehensive material data is input into the angle prediction model to obtain the opening angle and closing angle of the material. The angle prediction model is constructed through an artificial intelligence model.

6. The bulk material grab control system based on machine vision according to claim 5, characterized in that: The angle prediction model is constructed by an artificial intelligence model, including: Obtain a number of historical material labels, material sizes, bulk densities, and surface states, as well as their corresponding opening angles and closing angles; Integrate a number of historical material labels, material sizes, bulk densities, and surface states into a number of historical material comprehensive data; Divide a number of historical material comprehensive data and their corresponding opening angles and closing angles into training data, validation data, and test data; perform data preprocessing on the training data, validation data, and test data to obtain a training set, a validation set, and a test set; Select an artificial intelligence model as the basic model; Train the basic model with the training set, and adjust the learning rate and hyperparameters on the validation set to obtain a pre-trained model; Verify the pre-trained model on the test set, and finally obtain an angle prediction model with the input of material comprehensive data and the output of opening angle and closing angle.

7. The bulk material grab control system based on machine vision according to claim 1, characterized in that: The generation of the grab adjustment parameters according to the grab coordinates and the target coordinates includes: Obtain the target coordinates and the grab volume; Generate the grab adjustment height for grasping according to the target coordinates and the grab volume; Obtain the grab coordinates and the corresponding wind force magnitude in real time; Calculate the grab movement adjustment speed according to the grab coordinates and the corresponding wind force magnitude.

8. The bulk material grab control system based on machine vision according to claim 7, characterized in that: The generation of the grab adjustment height for grasping according to the target coordinates and the grab volume includes: Obtain the grab coordinates, target coordinates, grab volume ZDV, and grab height ZDG; When the vertical coordinate in the target coordinates is less than the grab height, set the grab adjustment height to the value corresponding to the grab height; Otherwise, construct the grab point cloud data according to the opening angle of the grab; Divide the grab point cloud data according to the standard unit height BDG, and calculate the volume TVi of each standard unit height through the voxelization method; where, i = 1, 2,..., N; N represents the total number after division, and i = 1 is the number corresponding to the top of the grab point cloud data; Obtain the i that satisfies the following conditions: ∑(TVi) < ZDV and ∑(TVi+1) ≥ ZDV; the ∑ summation range is [1, N]; When the volume of the grab point cloud data is less than ZDV, set the grab adjustment height to the value corresponding to the grab height; otherwise, calculate the grab adjustment height through the formula ZTG = BDG × (N - (i + 1)) + ZDG.

9. The bulk material grab control system based on machine vision according to claim 7, characterized in that: The calculation of the grab movement adjustment speed according to the grab coordinates and the corresponding wind force magnitude includes: Obtain the grab coordinates and the corresponding wind force magnitude FD, the grab adjustment height ZTG, and the corresponding wind force magnitude ZTFD in real time; When the vertical coordinate ZZ in the grab bucket coordinates is greater than the sum of the grab adjustment height and the adjustment threshold, the formula Calculate the grab moving adjustment speed ZYTS; Otherwise, the grab movement adjustment speed ZYTS is calculated by the formula ZYTS=β×BS×exp(-(α1×ZTG / DG+α2×ZTFD / DFD)); wherein α1 and α2 are weight coefficients, α1 and α2∈(0,1); β is the proportional coefficient, β∈(0,1); is the rounding symbol, DG is the unit height, DFD is the unit wind force, and BS is the standard speed.

10. The bulk material grab control system based on machine vision according to claim 1, characterized in that: The generation of the alarm signal according to the grab data includes: Obtain the grab coordinates and the two-dimensional plane area of the graspable area; Map the horizontal and vertical coordinates in the grab coordinates to the two-dimensional plane to obtain two-dimensional coordinates; When the two-dimensional coordinates exceed the two-dimensional plane area of the graspable area; generate a grab offset alarm signal; otherwise, do nothing.

Images