Telescopic AGV forklift based on visual guidance and operation method thereof

By integrating the depth camera and Faster R-CNN model on the AGV forklift, combined with the closed-loop stabilizer, the automatic identification and stable transportation of cable cargo is achieved, and the problem of insufficient flexibility in transporting cable cargo in the existing technology is solved, and the problem of AGV forklift transporting cable cargo is improved. Transport stability and identification accuracy are achieved.

CN119059462AInactive Publication Date: 2024-12-03GUILIN UNIVERSITY OF TECHNOLOGY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411401434.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-09
Publication Date
2024-12-03
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing AGV forklifts are not flexible enough to transport cable cargo, making it difficult to accurately identify the location of the cargo, resulting in the cargo being easily shedded or unstable transportation, increasing the time cost.

Method used

A visually guided retractable AGV forklift is designed, combining a depth camera and a Faster R-CNN model to achieve automatic identification and stable transportation of cable cargo through image recognition and closed-loop stabilizer.

Benefits of technology

It improves the stability of cable cargo transportation, ensures that the goods do not fall during transportation, reduces transportation time, and improves the flexibility and identification accuracy of AGV forklifts.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119059462A_ABST
    Figure CN119059462A_ABST
Patent Text Reader

Abstract

The invention discloses a telescopic AGV forklift based on visual guidance and an operation method thereof, and belongs to the technical field of machine learning. The telescopic AGV forklift comprises an AGV forklift body, a pallet fork driving frame, a telescopic cylindrical pallet fork, a driving type closed-loop stabilizer and a depth camera. The operation method comprises the following steps: training a prediction coordinate prediction model by using a Faster R-CNN model, so that the trained prediction coordinate prediction model can identify a collected cable cargo image and calculate coordinate data required for adjusting an alignment error; the telescopic AGV forklift achieves self-fine adjustment by predicting coordinate data fed back by the coordinate prediction model. According to the invention, the structure of the AGV forklift is improved, and the coordinate position of the cable goods is predicted through the training prediction model, so that the AGV forklift can automatically align the goods and complete stable transportation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and more particularly to a visually guided retractable AGV forklift and an operating method thereof. Background Art

[0002] With the rapid development of artificial intelligence technology, the integration of robots into industrial manufacturing to achieve automated production has undoubtedly become an inevitable trend of the times. AGV (Automated Guided Vehicle) forklift is an industrial robot that has functions such as automatic cruising, obstacle avoidance, cargo loading, and transportation. More and more cable manufacturers are beginning to introduce AGV forklifts, which can improve production efficiency on the one hand and save labor costs on the other. However, the flexibility of AGV forklifts is difficult to achieve manual operation, so how to accurately identify the location of cable cargo and transport it on pallets is the core issue that people are currently concerned about.

[0003] Since the cargo in the cable factory is cylindrical, the probability of cargo rolling is very high. At present, the solution to this problem is mostly to fix a baffle at a certain distance on the ground of the cable warehouse or use shelves of the same size to load the cables. In this way, the fixed coordinates can be given to the existing AGV forklift to transport the cargo. However, this pallet transportation method currently has the following shortcomings:

[0004] (1) The position and posture of the cable cargo must be fixed. If the deviation is too large, the AGV forklift will not be able to complete the loading and transportation of the cargo.

[0005] (2) If the cables transported by the existing AGV forklift are not loaded on shelves, the cable cargo is easily dropped due to the constraints of the AGV forklift fork shape; if the cables are loaded on shelves, the shelves must be unloaded when unloading, which increases the time cost.

[0006] Therefore, how to provide a visually guided retractable AGV forklift and an operating method thereof that can solve the above-mentioned problems is an issue that those skilled in the art urgently need to solve. Summary of the invention

[0007] In view of this, the present invention provides a retractable AGV forklift based on vision guidance and an operation method thereof. In view of the shortcomings and deficiencies of the prior art, the present invention improves the structure of the AGV forklift to achieve stable transportation of cylindrical cable cargoes, and then trains an image recognition model for identifying cable cargoes based on machine learning. Finally, the model is deployed and tested and loaded onto the AGV forklift.

[0008] In order to achieve the above object, the present invention provides the following technical solutions:

[0009] A retractable AGV forklift based on vision guidance, comprising: an AGV forklift body, a fork drive frame, a retractable cylindrical fork, a driven closed-loop stabilizer and a depth camera;

[0010] Among them, the fork driving frame is installed on the AGV forklift body, and the bottom of the fork driving frame is connected to the retractable cylindrical fork; the top periphery of the retractable cylindrical fork is the driven closed-loop stabilizer used to prevent the transported goods from falling off, and the driven closed-loop stabilizer is in a retracted and closed state when the goods are not lifted, and the depth camera for collecting images is installed at the center of the top circle of the retractable cylindrical fork, and the collected original RGBD cable cargo data is passed to the pre-built coordinate prediction model through the depth camera.

[0011] Optionally, a camera control module is also included, the depth camera is connected to the camera control module, and a driving pipeline is configured, the depth camera receives a startup stream, and controls the depth camera to collect original RGBD cable cargo image data.

[0012] Optionally, it also includes a control motor, which grinds the retractable cylindrical fork into a shape capable of bearing the weight of the cargo, performs a controllable closed-loop drive on the top of the grinded retractable cylindrical fork, installs the retractable cylindrical fork inside the chassis of the AGV forklift body and connects it to the control motor, so as to realize the extension and retraction of the retractable cylindrical fork from the inside to the outside.

[0013] A method for operating a retractable AGV forklift, comprising:

[0014] S200: Control the depth camera through the camera control module to collect the cable cargo image and use it as the original RGBD cable cargo image data;

[0015] S300: Use the Faster R-CNN model framework to train the model and obtain the coordinate prediction model;

[0016] S400: The original RGBD cable cargo image collected by S200 is recognized through the coordinate prediction model and the coordinate data is fed back. The retractable AGV forklift can automatically cruise to the side of the cable cargo and make fine adjustments to the alignment.

[0017] S100: Removing cylindrical cargo such as cables using retractable cylindrical forks.

[0018] Optionally, S300 includes:

[0019] S310: Prepare pictures of cable coil cargo, ensuring that the images cover various angles, lighting conditions and backgrounds;

[0020] S320: creating two categories of image annotations, namely cable loop cargo annotation and cable loop side circular annotation, to obtain annotation data;

[0021] S330: Convert the labeled data to COCO format, normalize the image pixel values, and perform data enhancement:

[0022] S340: Use the Faster R-CNN model as the pre-trained network framework and set the learning rate, batch size, and number of training rounds.

[0023] S350: importing the labeling data of the cargo with the labeling category of cable loop, obtaining the prediction result of the coordinate prediction model, and calculating the total loss;

[0024] S360: Import the annotation data with the annotation category of the side circle of the cable loop, convert the image of the side circle of the cable loop into a grayscale image, perform edge detection and Hough circle detection, output the difference coordinates between the current circle and the standard circle, and obtain the accuracy of the coordinate prediction model;

[0025] S370: Repeat the training from step S340 to S360 until the accuracy of the coordinate prediction model reaches the predicted value, and obtain the coordinate prediction model.

[0026] Optionally, after using the Faster R-CNN model to train the coordinate prediction model, the method further includes: exporting the trained coordinate prediction model and saving it in a format for deploying TensorFlow SavedModel for use in an actual AGV forklift application.

[0027] Optionally, S400 includes:

[0028] S420: The original RGBD cargo image data input stream is detected, and the coordinate prediction model is ready to receive data;

[0029] S430: The coordinate prediction model identifies the original RGBD cable cargo image data and calculates the coordinate data required for the current AGV forklift to cruise to the side;

[0030] S440: The retractable AGV forklift receives the feedback coordinate data and automatically cruises to the side of the cable cargo.

[0031] Optionally, S400 also includes:

[0032] S450: The coordinate prediction model uses the original RGBD cable cargo image data and calculates the coordinate data required to align the cargo side;

[0033] S460: The AGV forklift fine-tunes the fork through the feedback coordinate data;

[0034] S470: After fine-tuning, the circular side of the cargo is aligned with the retractable cylindrical fork, and the retractable cylindrical fork is controlled to perform the insertion operation. Closed-loop control is performed based on the detection feedback to prevent the cargo from slipping or position shifting.

[0035] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses and provides an operating method thereof, which has the following beneficial effects:

[0036] (1) Compared with the traditional AGV forklift, the new AGV forklift of the present invention is an improvement on the internal structure of the existing forklift. It is particularly suitable for the transportation of cylindrical goods such as cables in cable factories. The special forks can ensure that the goods do not fall during transportation, thereby increasing transportation stability.

[0037] (2) Compared with traditional AGV forklifts, the present invention trains the coordinate prediction model of the forklift based on Faster R-CNN. The prediction model automatically cruises to the location of the target goods through the real-time images collected by the depth camera, and then aligns the front position of the goods through the Hough circle monitoring algorithm to achieve self-fine-tuning. The model has a high degree of training, is not prone to overfitting problems, and has high recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0039] Figure 1 The figure is a schematic structural diagram of a new AGV forklift suitable for a cable factory according to the present invention.

[0040] Figure 2 The figure is a flow chart of the coordinate prediction model trained based on Faster R-CNN in the present invention.

[0041] Figure 3 The present invention is a flow chart of an operating method of a new AGV forklift suitable for a cable factory.

[0042] Markings in the figure:

[0043] 1-AGV forklift body, 2-fork drive frame, 3-retractable cylindrical fork, 4-driven closed-loop stabilizer, 5-depth camera. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] Embodiment 1:

[0046] On the one hand, see Figure 1 , the embodiment of the present invention discloses a visually guided retractable AGV forklift, comprising: an AGV forklift body 1, a fork drive frame 2, a retractable cylindrical fork 3, a driven closed-loop stabilizer 4 and a depth camera 5;

[0047] The AGV forklift body 1 is surrounded by a fork drive frame 2 that can slide up and down. The bottom of the drive frame 2 is connected to the cylindrical fork 3, which serves as the Y-axis pull-up drive when the fork 3 is aligned with the cargo for fine-tuning. When the fork 3 is not aligned with the cargo, it shrinks in the base to reduce the size of the forklift. The top periphery of the fork 3 is a driven closed-loop stabilizer 4 used to prevent the cargo from falling off during transportation. When the cargo is not lifted, the driven closed-loop stabilizer 4 is in a retracted and closed state. At the center of the top circle of the fork 3 is a depth camera 5 for collecting images. The depth camera 5 transmits the collected original RGBD cable cargo image data to the coordinate prediction model.

[0048] Specifically, it also includes a camera control module, the depth camera 5 is connected to the camera control module, and a driving pipeline is configured. The depth camera 5 receives the startup stream and controls the depth camera 5 to collect original RGBD cable cargo image data.

[0049] The AGV forklift also includes a control motor, which grinds the retractable cylindrical fork 3 into a shape capable of bearing the weight of the goods, performs a controllable closed-loop drive on the top of the grinded retractable cylindrical fork 3, installs the retractable cylindrical fork 3 inside the chassis of the AGV forklift body 1 and connects it to the control motor, so as to realize the retractable cylindrical fork 3 from the inside to the outside.

[0050] Specifically, a fork that can bear the weight of the cargo is polished into a cylindrical shape to be suitable for the transportation of cylindrical cable cargo, and then a depth camera 5 and a controllable closed-loop drive stabilizer are installed on the top of the polished fork to collect subsequent image data and ensure the stability of cargo transportation.

[0051] On the other hand, this embodiment also discloses a method for operating a retractable AGV forklift based on vision guidance, including:

[0052] S200: Control the depth camera 5 through the camera control module to collect the cable cargo image and use it as the original RGBD cable cargo image data;

[0053] S300: Use the Faster R-CNN model framework to train the model and obtain the coordinate prediction model;

[0054] S400: The original RGBD cable cargo image collected by S200 is recognized through the coordinate prediction model and the coordinate data is fed back. The retractable AGV forklift can automatically cruise to the side of the cable cargo and make fine adjustments to the alignment.

[0055] S100: Pick up cable-type cylindrical cargo by using the retractable cylindrical fork 3.

[0056] Specifically, S300 includes:

[0057] S310: Prepare pictures of cable coil cargo, ensuring that the images cover various angles, lighting conditions and backgrounds;

[0058] S320: creating two categories of image annotations, namely cable loop cargo annotation and cable loop side circular annotation, to obtain annotation data;

[0059] S330: Convert the labeled data to COCO format, normalize the image pixel values, and perform data enhancement:

[0060] S340: Use the Faster R-CNN model as the pre-trained network framework and set the learning rate, batch size, and number of training rounds.

[0061] S350: importing the labeling data of the cargo with the labeling category of cable loop, obtaining the prediction result of the coordinate prediction model, and calculating the total loss;

[0062] S360: Import the annotation data with the annotation category of the side circle of the cable loop, convert the image of the side circle of the cable loop into a grayscale image, perform edge detection and Hough circle detection, output the difference coordinates between the current circle and the standard circle, and obtain the accuracy of the coordinate prediction model;

[0063] S370: Repeat the training from step S340 to S360 until the accuracy of the coordinate prediction model reaches the predicted value.

[0064] Specifically, after using the Faster R-CNN model to train the coordinate prediction model, the method further includes: exporting the trained coordinate prediction model and saving it in a format for deploying TensorFlow SavedModel for use in actual AGV forklift applications.

[0065] Specifically, S400 includes:

[0066] S420: The original RGBD cable cargo image data input stream is detected, and the coordinate prediction model is ready to receive data;

[0067] S430: The coordinate prediction model calculates the coordinate data required for the current AGV forklift to cruise to the side through the image data. The AGV forklift automatically cruises to the side of the cable cargo after receiving the feedback coordinate data.

[0068] Specifically, S400 also includes:

[0069] S440: The coordinate prediction model again calculates the coordinate data required for aligning the side of the cargo through the captured image data of the side of the cable cargo;

[0070] S450: The AGV forklift fine-tunes the fork through the feedback coordinate data;

[0071] S460: After fine-tuning, the circular side of the cargo is aligned with the retractable cylindrical fork 3, and the retractable cylindrical fork 3 is controlled to perform an insertion operation. Closed-loop control is performed based on the detection feedback to prevent the cargo from slipping or position shifting.

[0072] Specifically, two coordinate prediction models for identifying cable cargo and cable cargo side circles are trained based on Faster R-CNN. The cable cargo coordinate prediction model is used to locate the specific cargo position in the cargo area. It is mainly done by preprocessing the labeled cable cargo image data and then importing it into the pre-trained model for learning; the cable cargo side coordinate prediction model is used for AGV forklifts to perform side alignment processing after finding the nearest cargo. It is mainly trained by using various types of labeled cable cargo side images. The model outputs the predicted coordinate data by comparing the received side circle data with the error of the standard circle; after the training is completed, the test set data is used to evaluate the performance of the model to see whether the F1 score and accuracy of the model in the test set meet the requirements.

[0073] Specifically, the accuracy A and F1 score are used as evaluation indicators, and the performance indicators of the model are evaluated. The specific definitions are as follows:

[0074]

[0075] Where A is the accuracy rate, TP represents the number of samples that are actually positive and correctly predicted as positive; FN represents the number of samples that are actually positive but incorrectly predicted as negative; FP represents the number of samples that are actually negative but incorrectly predicted as positive; TN represents the number of samples that are actually negative and correctly predicted as negative. The higher the F1 value, the better the performance of the classifier. If the F1 value is lower than 0.95, adjust and optimize the configuration parameters of the prediction model.

[0076] Specifically, the goods are aligned by adjusting the error parameters fed back by the coordinate prediction model. The error parameters are two-dimensional coordinates in the plane. The error in the x direction is adjusted by the left and right translation of the AGV forklift rollers. The error in the y direction is adjusted up and down by the drive frame on the AGV forklift. After the error adjustment is completed, the prediction model is used to analyze whether the goods are aligned. If the goods are aligned, the goods are lifted by the forks and the closed-loop stabilizer is opened to transport the goods.

[0077] Embodiment 2:

[0078] The technical solution of the embodiment of the present invention includes four aspects: fork design modification, image acquisition, training coordinate prediction model, and posture adjustment.

[0079] The fork design modification includes: designing the fork into a retractable cylindrical shape, making it suitable for picking up cable cylindrical cargo of different lengths;

[0080] Image acquisition includes: installing a depth camera 5 on the modified cylindrical fork, and controlling the depth camera 5 to collect raw RGBD data through a camera control module;

[0081] Training the coordinate prediction model includes: enabling the trained coordinate prediction model to calculate the coordinate data required for the current AGV forklift to cruise to the side and the coordinate data required to adjust the side alignment error by recognizing the original RGBD cable cargo image collected by S200;

[0082] Posture adjustment includes: using the coordinate data fed back by the coordinate prediction model, the AGV forklift can adjust itself according to the coordinates.

[0083] The steps for fork design modification are as follows:

[0084] Step S11: grinding a cylindrical fork that matches the size of the cable cargo drum slot into a shape that can bear the weight of the cargo;

[0085] Step S12: driving the polished fork tip in a controllable closed loop;

[0086] Step S13: installing the new fork inside the forklift chassis and connecting it to the forklift control motor so that it can be extended from the inside to the outside;

[0087] The image acquisition steps in step S2 are as follows:

[0088] Step S21: Install a depth camera 5 at the center of the circle at the top of the fork;

[0089] Step S22: Connect the depth camera 5 to the camera control module and configure the driving pipeline;

[0090] Step S23: Start the stream and collect the original RGBD image data;

[0091] like Figure 2 As shown, the training flow chart of the present invention is based on the Faster R-CNN training coordinate prediction model:

[0092] Step 1: Use the depth camera 5 to collect the original RGBD image data required for training the model;

[0093] Step 2: Preprocess the image data using pixel value normalization to ensure that each sample is processed at the same scale. The calculation formula is as follows:

[0094]

[0095] In the formula, x i Indicates the image pixel value that needs to be normalized max(x) indicates the maximum value of the image pixel, and min(x) indicates the minimum value of the image pixel;

[0096] Step 3: annotate the collected images, including marking bounding boxes or segmenting polygons for the original RGBD images of the cable cargo and the side circular images of the cable cargo, and then unify the XML format or TXT format into COCO format through Python script;

[0097] Step 4: Increase data diversity and avoid model overfitting by scaling, cropping, rotating, adjusting brightness, contrast, saturation, and randomly adding Gaussian noise or salt and pepper noise to the image;

[0098] Step 5: Divide all collected samples into training sets and test sets, with 70%-80% of the data used for training and 20%-30% of the data used for testing;

[0099] Step 6: Modify the configuration parameters of the model, including learning rate, batch size, and maximum number of iterations. The learning rate determines the update step of the model parameters and is set between 0.001 and 0.0001; the batch size is set to 32 or 64, and the update step calculation formula is as follows:

[0100]

[0101] Where a is the learning rate, is the gradient of the loss function.

[0102] Step 7: Load the pre-trained model to speed up training and improve the initial performance of the model;

[0103] Step 8: Use the data in the training set to train the model. During the training process, the model will be optimized according to the loss function and the weight parameters will be adjusted continuously to minimize the prediction error. The loss function calculation formula is as follows:

[0104]

[0105] In the formula, y i represents the true label, represents the predicted probability;

[0106] Step 9: After training, use the test set data to evaluate the performance of the model, build a confusion matrix to calculate the F1 score index and the accuracy A. The calculation formula is as follows:

[0107]

[0108] In the formula, TP represents the number of samples that are actually positive and correctly predicted as positive; FN represents the number of samples that are actually positive but incorrectly predicted as negative; FP represents the number of samples that are actually negative but incorrectly predicted as positive; TN represents the number of samples that are actually negative and correctly predicted as negative.

[0109] Step 10: Export the trained model and save it in the format of deploying TensorFlow SavedModel for use in the actual AGV forklift application.

[0110] like Figure 3 As shown, a flow chart of an operation method of a new AGV forklift applicable to a cable factory of the present invention includes the AGV forklift identifying goods and automatically fine-tuning:

[0111] Step 1: The AGV forklift first cruises to the cargo transportation area through a predetermined path planning and navigation algorithm, and locates a position close to the cable cargo;

[0112] Step 2: After approaching the cargo, the AGV forklift activates the depth camera 5 to capture the original RGBD image of the cable cargo;

[0113] Step 3: Feed the captured original RGBD image to the cargo coordinate prediction model. According to the adjusted coordinates output by the coordinate prediction model, the AGV forklift automatically adjusts its position and direction so that it can face the side of the cable cargo.

[0114] Step 4: The AGV forklift uses the depth camera 5 to capture the original RGBD image of the side of the cable cargo. After receiving the input original RGBD image, the coordinate prediction model determines whether the side is a standard circle.

[0115] First, use a Gaussian filter to denoise, calculate the gradient to get the edge strength and direction, and use non-maximum suppression to retain the strongest edge. The gradient amplitude and direction calculation formula are as follows:

[0116]

[0117] In the formula, Gy Represents the gradient of the image in the y direction, G x Represents the gradient of the image in the x direction;

[0118] Hough circle transform is applied to the detected edge map, and the circle is identified by identifying edge points that conform to the circle in the parameter space through a voting mechanism. The calculation parameter equation of the circle is as follows:

[0119] (x) 2 +(yb) 2 =r 2 (8)

[0120] In the formula, (a, b) represents the coordinates of the center of the circle, and r represents the radius of the circle;

[0121] The Hough transform determines the center and radius of the circle by finding the maximum voting value in the parameter space (a, b, r);

[0122] Step 5: If the judgment result does not meet the error range of the standard circle, the coordinate prediction model calculates the required adjustment amount of the fork in the X-axis and Y-axis directions so that the fork can be aligned with the standard circle of the cable cargo. The adjustment amount is based on the deviation between the detected center of the circle and the standard center of the circle;

[0123] Step six: After fine-tuning, the circular side of the cargo is aligned with the fork. The AGV forklift drives the fork to accurately insert the cable cargo, and performs closed-loop control based on the detection feedback to prevent the cargo from slipping or shifting.

[0124] Through the above technical solution, the present invention improves the structure of the AGV forklift, and predicts the coordinate position of cable cargo by training a prediction model, so that the AGV forklift can automatically align the cargo and complete stable transportation.

[0125] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0126] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A visually guided retractable AGV forklift, characterized in that: include: AGV forklift body, fork drive frame, retractable cylindrical fork, driven closed-loop stabilizer and depth camera; Among them, the fork driving frame is installed on the AGV forklift body, and the bottom of the fork driving frame is connected to the retractable cylindrical fork; the top periphery of the retractable cylindrical fork is the driven closed-loop stabilizer used to prevent the transported goods from falling off, and the driven closed-loop stabilizer is in a retracted and closed state when the goods are not lifted, and the depth camera for collecting images is installed at the top center of the retractable cylindrical fork, and the collected original RGBD cable cargo image data is passed to the pre-built coordinate prediction model through the depth camera.

2. A visually guided retractable AGV forklift according to claim 1, characterized in that: It also includes a camera control module, the depth camera is connected to the camera control module, and a driving pipeline is configured. The depth camera receives a startup stream and controls the depth camera to collect original RGBD cable cargo image data.

3. The vision-guided retractable AGV forklift according to claim 1, characterized in that: It also includes a control motor, which grinds the retractable cylindrical fork into a shape capable of bearing the weight of the goods, performs a controllable closed-loop drive on the top of the grinded retractable cylindrical fork, installs the retractable cylindrical fork inside the chassis of the AGV forklift body and connects it to the control motor, so as to realize the retractable cylindrical fork from the inside to the outside.

4. A method for operating a telescopic AGV forklift based on vision guidance as claimed in any one of claims 1 to 3, characterized in that: include: S200: Control the depth camera through the camera control module to collect the cable cargo image and use it as the original RGBD cable cargo image data; S300: Use the Faster R-CNN model framework to train the model and obtain the coordinate prediction model; S400: The original RGBD cable cargo image collected by S200 is recognized through the coordinate prediction model and the coordinate data is fed back. The retractable AGV forklift can automatically cruise to the side of the cable cargo and make fine adjustments to the alignment. S100: Removing cylindrical cargo such as cables using retractable cylindrical forks.

5. The operating method according to claim 4, characterized in that: S300 includes: S310: Prepare pictures of cable coil cargo, ensuring that the images cover various angles, lighting conditions and backgrounds; S320: creating two categories of image annotations, namely cable loop cargo annotation and cable loop side circular annotation, to obtain annotation data; S330: Convert the labeled data to COCO format, normalize the image pixel values, and perform data enhancement: S340: Use the Faster R-CNN model as the pre-trained network framework and set the learning rate, batch size, and number of training rounds. S350: importing the labeling data of the cargo with the labeling category of cable loop, obtaining the prediction result of the coordinate prediction model, and calculating the total loss; S360: Import the annotation data with the annotation category of the side circle of the cable loop, convert the image of the side circle of the cable loop into a grayscale image, perform edge detection and Hough circle detection, output the difference coordinates between the current circle and the standard circle, and obtain the accuracy of the coordinate prediction model; S370: Repeat the training from step S340 to S360 until the accuracy of the coordinate prediction model reaches the predicted value, and obtain the coordinate prediction model.

6. The operating method according to claim 4, characterized in that: After using the Faster R-CNN model to train the coordinate prediction model, it also includes: exporting the trained coordinate prediction model and saving it in the format of deploying TensorFlowSavedModel for use in actual AGV forklift applications.

7. The operating method according to claim 4, characterized in that: S400 includes: S420: The original RGBD cargo image data input stream is detected, and the coordinate prediction model is ready to receive data; S430: The coordinate prediction model identifies the original RGBD cable cargo image data and calculates the coordinate data required for the current AGV forklift to cruise to the side; S440: The retractable AGV forklift receives the feedback coordinate data and automatically cruises to the side of the cable cargo.

8. The operating method according to claim 7, characterized in that: The S400 also includes: S450: The coordinate prediction model uses the original RGBD cable cargo image data and calculates the coordinate data required to align the cargo side; S460: The AGV forklift fine-tunes the fork through the feedback coordinate data; S470: After fine-tuning, the circular side of the cargo is aligned with the retractable cylindrical fork, and the retractable cylindrical fork is controlled to perform the insertion operation. Closed-loop control is performed based on the detection feedback to prevent the cargo from slipping or position shifting.