Wood intelligent tallying system and method based on visual identification
Through the combination of triple-eye camera and convolutional neural network, automatic identification and tracking of door machines and forklift grabs is achieved, which solves the problem of low efficiency of cargo unloading in the existing technology, and improves cargo efficiency and recognition accuracy.
Patent Information
- Application Number
- CN202411096789.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-12
- Publication Date
- 2024-12-03
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In the existing wood ship unloading operations, the operating path radius of the door machine grab is large, resulting in low cargo-tipping efficiency. The camera focal length needs to be enlarged to identify the front of the wood, which cannot further improve cargo-tipping efficiency.
Using a triple-eye camera (one detail camera and two wide-angle cameras) with recognition algorithm, it realizes automatic identification and tracking of door machine grabs and forklift grabs, image classification and feature extraction are performed through convolutional neural network, and the camera steering and Zoom control to achieve automatic screenshot and recognition.
It improves the efficiency of cargo cleaning in wood unloading, reduces manual intervention, and realizes automatic tracking and cargo cleaning in cabin operations, improving identification accuracy and efficiency.
Smart Images

Figure CN119068391A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the field of timber unloading and tallying, and in particular to an intelligent timber tallying system based on visual recognition. Background Art
[0002] With the steady improvement of the quality of timber tallying, the timber unloading process and technology are also constantly improving and improving. The previous ship-shore-truck operation mode has gradually been adjusted to a ship-truck direct loading mode. This reduces the loading and unloading links in the process. On the one hand, it can reduce the damage rate of timber caused by the loading and unloading process. On the other hand, after the operation process matures, it also improves the on-site operation efficiency and reduces the investment in on-site machinery and manpower.
[0003] The existing timber unloading operation is usually equipped with a set of timber intelligent tallying system to assist in the unloading of timber. During the process, the timber is transported to the designated location by forklift loading. However, the operation path of the gantry crane grab is to grab the timber from the cabin, and then rotate it from the cabin to the transfer vehicle under the gantry crane on the shore. The radius of the entire operation track is large. When capturing the front cross-section of the timber, the focal length of the camera needs to be enlarged to clearly view and identify the number of timber roots, resulting in the inability to further improve the tallying efficiency. Summary of the invention
[0004] The purpose of the present invention is to address the deficiencies of the prior art and provide a wood intelligent tallying system and method based on visual recognition. Automatic tallying is achieved by using a three-eye camera (including a detail camera and two wide-angle cameras) in combination with a recognition algorithm, a gantry crane and a forklift, wherein a wide-angle camera in the three-eye camera is used to define the recognition range to identify whether there is a gantry crane grab within the range and calibrate the position of the gantry crane grab; the detail camera in the three-eye camera will automatically track the gantry crane grab in combination with the calibrated gantry crane grab position in the aforementioned panoramic camera; the last wide-angle camera in the three-eye camera is used to identify the forklift grab in the picture, and automatically take screenshots and identify it. In summary, the three-eye camera is used to achieve full-process automatic tracking and tallying of a cabin operation, so as to achieve the purpose of improving tallying efficiency.
[0005] The technical solution of the present invention is as follows:
[0006] The system includes a camera module for acquiring working scenes, and an algorithm module for detecting and analyzing the acquired images;
[0007] The wood intelligent tallying system is also equipped with model training for optimizing algorithms, sorting equipment for tallying wood, and an alarm module for warning of abnormal working conditions;
[0008] The camera module includes three cameras and corresponding power supply and network;
[0009] The algorithm module includes assembly language for writing code and convolutional neural network for identifying the state of wood;
[0010] The sorting equipment includes a door machine and a grab machine, as well as clamps installed at the working ends of the two.
[0011] Preferably, the power source comprises a main battery and a backup battery, and both the main battery and the backup battery are composed of lithium batteries.
[0012] Preferably, the network includes switches for transmitting signals and routers for providing network connections.
[0013] Preferably, the model training is combined with a convolutional neural network, and the model training includes a data set and a test set.
[0014] Preferably, the model training also includes a data preprocessing module and a code module, and the code module is combined with assembly language.
[0015] Preferably, the convolutional neural network includes a data input layer, a convolution calculation layer, an excitation layer, a pooling layer, and a fully connected layer.
[0016] A wood intelligent tallying method based on visual recognition, the method is applied to claims 1-6, and the method steps are as follows:
[0017] S1, the gantry crane starts unloading the ship, grabs the wood through the grab bucket, and the camera moves to the preset point;
[0018] S2: The gantry crane grabs the machine out of the cabin, and the camera automatically tracks, captures and identifies it;
[0019] S3, during the recognition process, the algorithm module and model training are used to determine whether the wood is positive and the amount of wood;
[0020] S4. The operation screen is automatically saved, and after the wood is loaded, it will be checked for confirmation, thus completing the operation.
[0021] Beneficial effects of the present invention:
[0022] Through target detection technology and wide-angle video stream recognition, the position of the gantry crane grab in the video surveillance is determined, and the detection results are analyzed by algorithm module. At the same time, image classification technology is used to distinguish whether the identified log end face is positive or not. According to the orientation of the gantry crane grab in the picture, the camera is turned and zoomed to track the gantry crane grab, automatically capture the positive section of the gantry crane grab, and automatically identify and save it. In the process, corresponding model training is required to determine the appropriate classification system, feature extraction, selection of good training samples, image preprocessing, selection of appropriate classification methods, classification post-processing, and finally the overall accuracy is evaluated to realize automatic tracking, automatic screenshot, and automatic recognition functions to achieve the purpose of improving tallying efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0024] Figure 1 It is a schematic diagram of the system structure of the present invention.
[0025] Figure 2 Schematic diagram of a camera module of the present invention.
[0026] Figure 3 It is a schematic diagram of the algorithm module of the present invention.
[0027] Figure 4 It is a schematic diagram of the finishing equipment of the present invention.
[0028] Figure 5 Schematic diagram of model training of the present invention
[0029] Figure 6 Schematic diagram of the convolutional neural network of the present invention
[0030] Figure 7 Schematic diagram of the process of the present invention
[0031] Figure 8 Schematic diagram of the method of the present invention DETAILED DESCRIPTION
[0032] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0033] To further understand the present invention, the present invention will be described in detail below with reference to the accompanying drawings.
[0034] Combination Figure 1-4 The system includes a camera module for acquiring a working scene, and an algorithm module for detecting and analyzing the acquired picture;
[0035] The wood intelligent tallying system is also equipped with model training for optimizing algorithms, sorting equipment for tallying wood, and an alarm module for warning of abnormal working conditions;
[0036] The camera module includes three cameras, as well as corresponding power supply and network;
[0037] The algorithm module includes assembly language for writing code and convolutional neural network for identifying the state of wood;
[0038] The sorting equipment includes a door machine and a grab machine, as well as clamps installed at the working ends of the two.
[0039] In this embodiment, the gantry crane grabber needs to have automatic tracking technology in the camera module after grabbing the wood. This technology uses computer vision technology to perform real-time dynamic monitoring of the on-site operation scene. It recognizes the gantry crane grabber in the video stream and combines the basic control functions of the camera module to achieve joint control. The camera module includes a three-eye camera, which is installed movably to allow for more flexible adjustment of its position and shooting angle. Therefore, the three-eye camera needs to be equipped with a corresponding power supply.
[0040] First, the target detection technology is used to identify the position of the gantry crane grab in the video surveillance through wide-angle video stream recognition, and the position information is calibrated and transmitted to the detail camera; the detail camera uses the position information to realize automatic tracking of the gantry crane grab, and uses image classification technology to distinguish whether the identified log end face is positive or not. According to the orientation of the gantry crane grab in the picture, the camera is turned and zoomed, so as to achieve the functions of tracking the gantry crane grab, automatically capturing the positive cross section of the gantry crane grab, and automatically identifying and saving it;
[0041] In the above process, it is necessary to use the corresponding model training, determine the appropriate classification system, feature extraction, select good training samples (collection of grab samples at different distances and angles), image preprocessing and selection of appropriate classification methods, classification post-processing, and finally evaluate the overall accuracy. In this technology, the input is usually an image of a specific object (a gantry grab with logs to be classified), and the output is a predicted class that defines and matches the input object (the type of gantry grab);
[0042] The automatic tracking and automatic identification designed above, combined with the existing intelligent timber tallying system, can be directly adapted to the existing operation process based on the existing tallying system functions;
[0043] Combination Figure 2 , the power supply adopts a main battery and a backup battery, and the main battery and the backup battery are both composed of lithium batteries;
[0044] In this embodiment, the lithium battery is actually a lithium-sulfur battery, which is a battery that uses lithium ions to transfer back and forth between positive and negative electrodes. Compared with other types of batteries, lithium batteries have higher energy density, can provide longer use time or smaller volume and weight, and have longer cycle life, and can withstand multiple charge and discharge without failure. Compared with other rechargeable battery models, lithium-sulfur batteries can maintain charge for a longer time, support fast charging, and can restore power more quickly. The use of this type of lithium-sulfur battery can effectively ensure that the camera module can work for a long time.
[0045] Combination Figure 2 , the network includes a switch for transmitting signals and a router for providing network connection;
[0046] In this embodiment, the switch mainly functions to forward and exchange data packets within the local area network so that different devices can communicate with each other, so that the camera module can send the acquired information to the algorithm module. At the same time, the switch can quickly decide which specific port to forward the data packet to according to the destination address of the data packet, thereby realizing direct communication between target devices, and can avoid network congestion and improve data transmission efficiency through learning and filtering mechanisms. The switch can use a managed switch, which has management and configuration functions and can perform advanced configuration and monitoring through a command line interface or a graphical user interface, and is suitable for medium and large enterprise networks.
[0047] Furthermore, the router transfers data packets between different networks according to the destination address, allows devices in different networks to communicate, and can connect to multiple networks. Through routing selection algorithms, such as distance vector algorithm, link state algorithm, etc., the best path is selected to transmit the data packet from the source network to the target network, thereby avoiding the time difference between the picture taken by the camera module and the actual situation, and enabling the camera module to pass the captured picture to the algorithm module in real time. At the same time, the router can protect the network from unauthorized access and attack through the firewall function and security policy. The router can specifically select a wireless router, which has an integrated wireless LAN function, allowing wireless devices to connect to a wired network and access the Internet, providing convenience during installation and flexibility during use.
[0048] Combination Figure 3 , Figure 5 and Figure 6 , the model training is combined with a convolutional neural network, and the model training includes a data set and a test set;
[0049] In this embodiment, in order to realize the automatic tracking function of intelligent timber tallying, it is necessary to collect samples of the operating gantry crane grabber (data set integration) in the on-site environment of the berth gantry crane loading, and conduct multi-category model training (test set training) to achieve the effect of automatic tracking and automatic capture. In order to better and more accurately realize the accuracy of automatic tracking, automatic capture and automatic recognition, and reduce manual intervention, the current log recognition model can be further sampled and trained according to the scene angle of automatic tracking to improve the accuracy of wood recognition and reduce the misrecognition rate in the tracking scene. The model training mainly includes two parts: the model training of the gantry crane grabber and the wood model training in the gantry crane grabber;
[0050] The system needs to be able to automatically track the door machine for the currently opened video channel. When multiple door machines appear in the monitoring screen, priority tracking can be performed according to the operating area (left or right) set by the user. The current technology uses a single ball camera with an "AI recognition + camera" control algorithm mode. In order to achieve better tracking effects, this solution uses a three-eye camera. This solution includes the new SDK function adaptation of the three cameras of the three-eye camera to be deployed and tested, the joint debugging test of the new camera, and the algorithm design test content for the new camera. If the new camera test is successful, two different versions of the camera tracking algorithm can be provided to achieve automatic tracking of different camera scenes on site. Functional coverage.
[0051] Combined diagram Figure 3 , Figure 5 and Figure 6 , the model training also includes a data preprocessing module and a code module, and the code module is combined with the assembly language;
[0052] In this embodiment, the assembly language uses Java as the main programming language to implement the backend functions. The implementation ideas and steps of the data set preprocessing process are as follows:
[0053] Since the data set after stacking each frame of the camera module is too large, we first extract part of a frame of the picture as the training set and the test set, copy the captured image to the training, verification and test directories, and then normalize the image to make it resistant to the attack of geometric transformation. It can find the invariants in the image, so as to know that these images are originally the same or a series, and accelerate the convergence of the training network. At the same time, the image data needs to be enhanced, because a lot of data will be lost when processing the image, so data enhancement processing is required;
[0054] Furthermore, model training also includes the implementation ideas and steps of the model construction, training, prediction and evaluation process. First, build a sequential model, set up the convolution layer, maximum pooling layer, fully connected layer, etc. Process the divided training set test set, etc., that is, multiply it to the entire image before performing other processing and set this value to a number between 0 and 1, and build relevant data sets, determine the image size, the model to be used, etc., and then train the model;
[0055] When the generator returns the generator data, an epoch ends and the number of rounds for the next data iteration is executed. When the generator of the validation set is a generator, this parameter specifies the number of times the generator of the validation set returns. When training the model, the best critical point of the model may be missed, that is, when the maximum accuracy is reached, the accuracy of the test set will decrease, and overfitting will occur.
[0056] Furthermore, in order to prevent or reduce overfitting, the second model uses Dropout. During training, a portion of the data of a unit in a certain layer of the neural network (excluding the units in the output layer) is randomly lost. In fact, it is equivalent to creating a lot of new random samples, so as to increase the sample size and reduce the number of features to prevent overfitting.
[0057] Furthermore, the vgg16 network is used at the same time. The VGG16 network consists of 13 convolutional layers + 3 fully connected layers. The outstanding feature of VGG16 is simplicity. The convolutional layers all use the same convolution kernel parameters, and the pooling layers all use the same pooling kernel parameters.
[0058] This system uses a convolutional neural network, which has three basic layers: convolution layer (convolution kernel), pooling layer, and fully connected layer. The convolution process uses a convolution kernel to continuously scan the pixel matrix at each layer according to the step size. The value scanned each time will be multiplied by the number at the corresponding position in the convolution kernel, and then added and summed. The obtained value will generate a new matrix. The convolution kernel is equivalent to a filter in the convolution operation, which is used to extract the features of the image. After the feature extraction, a feature map will be obtained. The pooling operation is equivalent to the dimensionality reduction operation. There are maximum pooling and average pooling. The maximum pooling is used this time. The second model VGG16 network is composed of 13 convolution layers + 3 fully connected layers. The Conv2D convolution layer adds 0s around the image matrix.
[0059] Combination Figure 6 , the convolutional neural network includes a data input layer, a convolution calculation layer, an excitation layer, a pooling layer, and a fully connected layer;
[0060] In this embodiment, the data input layer: mainly performs preprocessing on the original image data, such as normalization;
[0061] Convolution calculation layer: This layer is the most important layer of the convolutional neural network, which calculates the matrix;
[0062] Excitation layer: Perform nonlinear mapping on the output of the convolution layer;
[0063] Pooling layer: The main function of the pooling layer is to compress the image;
[0064] Fully connected layer: All neurons between two layers are connected by weights. Usually, the fully connected layer is at the end of the convolutional neural network.
[0065] Combination Figure 7 and Figure 8 , a wood intelligent tallying method based on visual recognition, the method is applied to claims 1-6, and the method steps are as follows:
[0066] S1, the gantry crane starts unloading the ship, grabs the wood through the grab bucket, and the camera moves to the preset point;
[0067] S2: The gantry crane grabs the machine out of the cabin, and the camera automatically tracks, captures and identifies it;
[0068] S3, during the recognition process, the algorithm module and model training are used to determine whether the wood is positive and the amount of wood;
[0069] S4, automatically save the operation screen, and confirm after the wood is loaded, thus completing the operation;
[0070] In the above method S2, during the automatic tracking process, the video stream automatic screenshot recognition function will be used in conjunction with the automatic tracking function. When the wood cross section of the gantry crane grab is facing the camera screen during the tracking process, the camera is controlled to zoom in in time, and the enlarged front cross section is automatically captured and the number of logs in the grab is automatically identified. The function of automatically saving the result of the automatic screenshot recognition is provided, and the automatic saving function is a user-selectable mode; and the user is supported to set the time interval for automatic saving according to different operation scenarios and operation speeds;
[0071] When the user adopts the automatic mode of automatic tracking and automatic saving, the system can alarm through the alarm module if it fails to effectively capture the front screenshot of the gantry crane grabber, and reminds manual confirmation of whether there are any missed identifications in the on-site tallying situation. The reminder logic is: if the moving distance of the gantry crane grabber is continuously identified, but the front screenshot of the gantry crane grabber cannot be captured within the time, angle, etc. set by the user, an alarm reminder will be issued. The alarm module can use a buzzer and a signal light as warning signals.
Claims
1. The intelligent timber tallying system based on visual recognition is characterized by: The system includes a camera module for acquiring working scenes, and an algorithm module for detecting and analyzing the acquired images; The wood intelligent tallying system is also equipped with model training for optimizing algorithms, sorting equipment for tallying wood, and an alarm module for warning of abnormal working conditions; The camera module includes three cameras, as well as corresponding power supply and network; The algorithm module includes assembly language for writing code and convolutional neural network for identifying the state of wood; The sorting equipment includes a door machine and a grab machine, as well as clamps installed at the working ends of the two.
2. The intelligent timber tallying system based on visual recognition according to claim 1 is characterized in that: The power source adopts a main battery and a backup battery, and both the main battery and the backup battery are composed of lithium batteries.
3. The intelligent timber tallying system based on visual recognition according to claim 1 is characterized in that: The network includes switches for transmitting signals and routers for providing network connections.
4. The intelligent timber tallying system based on visual recognition according to claim 1 is characterized in that: The model training is combined with a convolutional neural network, and the model training includes a data set and a test set.
5. The intelligent timber tallying system based on visual recognition according to claim 4 is characterized in that: The model training also includes a data preprocessing module and a code module, and the code module is combined with the assembly language.
6. The intelligent timber tallying system based on visual recognition according to claim 4 is characterized in that: The convolutional neural network includes a data input layer, a convolution calculation layer, an excitation layer, a pooling layer, and a fully connected layer.
7. A wood intelligent tallying method based on visual recognition, the method is applied to claims 1-6, and the method steps are as follows: S1, the gantry crane starts unloading the ship, grabs the wood through the grab bucket, and the camera moves to the preset point; S2: The gantry crane grabs the machine out of the cabin, and the camera automatically tracks, captures and identifies it; S3, during the recognition process, the algorithm module and model training are used to determine whether the wood is positive and the amount of wood; S4. The operation screen is automatically saved, and after the wood is loaded, it will be checked for confirmation, thus completing the operation.
