Ton bag processing method and device in automatic production

Abstract

The invention discloses a processing method and device for ton bags in automatic production. The method comprises the steps that when a ton bag grabbing instruction is received, a ton bag image of a target area is collected through an image collecting component; inputting the ton bag image into a ton bag identification model, and processing the ton bag image by using the ton bag identification model to obtain target position information of a ton bag lug of a target ton bag corresponding to the ton bag image; a ton bag grabbing instruction is generated according to the target position information; and the ton bag grabbing instruction is sent to an industrial personal computer, so that the industrial personal computer is used for controlling an industrial robot to grab a target ton bag according to the ton bag grabbing instruction, and the target ton bag is placed at a designated position. The technical problem that a grabbing robot used in industrial automatic production in the related technology has large limitation and cannot adapt to complex production conditions is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of automated production, and in particular to a method and device for processing ton bags in automated production. Background Art

[0002] Traditional loading and unloading tasks are usually performed by workers. Nowadays, the loading and unloading industry is vigorously introducing mechanical equipment to reduce the burden of manual handling. Applying machine vision-based robotic arms to the working environment, first of all, from a long-term perspective, can increase benefits and reduce labor costs in loading and unloading; secondly, machines can reduce the incidence of accidents, and using robots to replace workers is a feasible solution.

[0003] At present, the most widely used grasping robots in industrial production lines are sorting robots, which often need to use visual sensors, which are equivalent to human eyes, to locate and analyze target objects. However, current grasping robots have great limitations and cannot adapt to more complex production conditions.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a method and device for handling ton bags in automated production, so as to at least solve the technical problem that the grasping robots used in industrial automated production in the related technology have great limitations and cannot adapt to relatively complex production conditions.

[0006] According to one aspect of an embodiment of the present invention, a method for processing ton bags in automated production is provided, comprising: upon receiving a ton bag grabbing instruction, acquiring a ton bag image of a target area through an image acquisition component, wherein the target area is an area where ton bags are placed; inputting the ton bag image into a ton bag recognition model, so as to process the ton bag image using the ton bag recognition model, so as to obtain target position information of ton bag ears of a target ton bag corresponding to the ton bag image, wherein the ton bag recognition model is a model trained by machine learning using multiple sets of training data, each of the multiple sets of training data comprising: a sample ton bag image and position information of a sample ton bag ears corresponding to the sample ton bag image; generating a ton bag grabbing instruction according to the target position information; and sending the ton bag grabbing instruction to an industrial computer, so as to use the industrial computer to control an industrial robot to grab the target ton bag according to the ton bag grabbing instruction, and place the target ton bag at a designated position.

[0007] Optionally, when a ton bag grabbing instruction is received, the ton bag image of the target area is collected by an image acquisition component, including: generating the ton bag grabbing instruction when a sensor arranged at the image acquisition component detects that the target ton bag is transported to the area where the image acquisition component is located; and sending the ton bag grabbing instruction to the image acquisition component to use the image acquisition component to collect the ton bag image of the target area.

[0008] Optionally, before inputting the ton bag image into a ton bag recognition model to process the ton bag image using the ton bag recognition model to obtain the target position information of the ton bag ears of the target ton bag corresponding to the ton bag image, the method for processing ton bags in automated production also includes: acquiring a plurality of the sample ton bag images and the position information of a plurality of the sample ton bag ears corresponding to the plurality of the sample ton bag images; training a plurality of groups of the training data including the plurality of the sample ton bag images and the position information of a plurality of the sample ton bag ears to obtain the ton bag recognition model.

[0009] Optionally, training is performed on multiple groups of the training data including multiple sample big bag images and multiple location information of the bag ears of the sample big bag to obtain the big bag recognition model, including: acquiring multiple historical big bag images acquired by an image acquisition device at different angles and under different lighting conditions; preprocessing the multiple historical big bag images to obtain multiple sample big bag images, wherein the preprocessing includes: scaling processing, translation processing, rotation processing and radial change processing; annotating the multiple sample big bag images to obtain the location information of the bag ears of the multiple sample big bag.

[0010] Optionally, training is performed on a plurality of groups of the training data including a plurality of the sample ton bag images and a plurality of the position information of the bag ears of the sample ton bag to obtain the ton bag recognition model, including: determining that the basic network model of the ton bag recognition model is the target detection model YOLOv5; training the target detection model YOLOv5 using a plurality of the training data including a plurality of the sample ton bag images and a plurality of the position information of the bag ears of the sample ton bag to obtain the ton bag recognition model.

[0011] According to another aspect of an embodiment of the present invention, there is also provided a device for processing ton bags in automated production, comprising: an acquisition module, for acquiring a ton bag image of a target area through an image acquisition component when a ton bag grabbing instruction is received, wherein the target area is an area where ton bags are placed; a first processing module, for inputting the ton bag image into a ton bag recognition model, so as to process the ton bag image using the ton bag recognition model, so as to obtain target position information of ton bag ears of a target ton bag corresponding to the ton bag image, wherein the ton bag recognition model is a model obtained by machine learning training using multiple sets of training data, each of the multiple sets of training data comprising: a sample ton bag image and position information of a sample ton bag ears corresponding to the sample ton bag image; a generation module, for generating a ton bag grabbing instruction according to the target position information; and a second processing module, for sending the ton bag grabbing instruction to an industrial computer, so as to use the industrial computer to control an industrial robot to grab the target ton bag according to the ton bag grabbing instruction, and place the target ton bag at a designated position.

[0012] Optionally, the acquisition module includes: a generating unit, used to generate the ton bag grabbing instruction when the sensor arranged at the image acquisition component detects that the target ton bag is transported to the area where the image acquisition component is located; and a sending unit, used to send the ton bag grabbing instruction to the image acquisition component, so as to use the image acquisition component to capture the ton bag image of the target area.

[0013] Optionally, the device for processing ton bags in the automated production also includes: an acquisition module, used to acquire a plurality of sample ton bag images and position information of a plurality of sample ton bag ears corresponding to the plurality of sample ton bag images before inputting the ton bag image into the ton bag recognition model to process the ton bag image using the ton bag recognition model to obtain the target position information of the ton bag ears of the target ton bag corresponding to the ton bag image; a training module, used to train a plurality of groups of the training data including a plurality of the sample ton bag images and position information of a plurality of the sample ton bag ears to obtain the ton bag recognition model.

[0014] Optionally, the training module includes: an acquisition unit, used to acquire multiple historical ton bag images acquired by an image acquisition device at different angles and under different lighting conditions; a preprocessing unit, used to preprocess the multiple historical ton bag images to obtain multiple sample ton bag images, wherein the preprocessing includes: scaling processing, translation processing, rotation processing and radiation change processing; a labeling unit, used to label the multiple sample ton bag images to obtain the position information of the bag ears of the multiple sample ton bags.

[0015] Optionally, the training module includes: a determination unit, used to determine that the basic network model of the ton bag recognition model is the target detection model YOLOv5; a training unit, used to train the target detection model YOLOv5 using multiple groups of training data including multiple sample ton bag images and multiple position information of the sample ton bag ears to obtain the ton bag recognition model.

[0016] According to another aspect of an embodiment of the present invention, a system for processing ton bags in automated production is provided. The system for processing ton bags in automated production uses any one of the methods for processing ton bags in automated production described above.

[0017] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is further provided, the computer-readable storage medium comprising a stored program, wherein the program executes any one of the above-mentioned methods for processing ton bags in automated production.

[0018] According to another aspect of an embodiment of the present invention, a processor is further provided, wherein the processor is used to run a program, wherein when the program is run, any one of the above-mentioned methods for processing ton bags in automated production is executed.

[0019] According to another aspect of an embodiment of the present invention, a computer program product is provided, comprising computer instructions, which, when executed by a processor, execute any one of the above-mentioned methods for processing ton bags in automated production.

[0020] In an embodiment of the present invention, when a ton bag grabbing instruction is received, a ton bag image of a target area is collected by an image acquisition component, wherein the target area is an area where ton bags are placed; the ton bag image is input into a ton bag recognition model, so that the ton bag image is processed by using the ton bag recognition model to obtain target position information of ton bag ears of a target ton bag corresponding to the ton bag image, wherein the ton bag recognition model is a model trained by machine learning using multiple sets of training data, and each of the multiple sets of training data includes: a sample ton bag image and position information of a sample ton bag ears corresponding to the sample ton bag image; a ton bag grabbing instruction is generated according to the target position information; the ton bag grabbing instruction is sent to an industrial computer, so that the industrial computer is used to control an industrial robot to grab the target ton bag according to the ton bag grabbing instruction, and place the target ton bag at a designated position. Through the technical solution provided by the present invention, the purpose of utilizing and improving the YOLOv5 algorithm, identifying ton bags through high-definition cameras, and then using a robotic arm to grab and unload them is achieved, thereby achieving the technical effect of efficiently and accurately identifying and grabbing ton bags, improving the flexibility and adaptability of production, and further solving the technical problem that the grabbing robots used in industrial automation production in related technologies have great limitations and cannot adapt to more complex production conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0022] Figure 1 This is a hardware structure block diagram of a mobile terminal for a method for processing ton bags in automated production according to an embodiment of the present invention;

[0023] Figure 2 is a flow chart of a method for processing ton bags in automated production according to an embodiment of the present invention;

[0024] Figure 3 is a flow chart of a method for processing ton bags in optional automated production according to an embodiment of the present invention;

[0025] Figure 4 It is an overall structural diagram of a method for processing ton bags in automated production according to an embodiment of the present invention;

[0026] Figure 5 is a data annotation schematic diagram of a method for processing ton bags in automated production according to an embodiment of the present invention;

[0027] Figure 6 is a data expansion schematic diagram of a method for processing ton bags in automated production according to an embodiment of the present invention;

[0028] Figure 7 Schematic diagram of a device for handling ton bags in automated production according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] As introduced in the background technology, the grabbing robots used in industrial automation production in the related art have great limitations and cannot adapt to the technical problems of more complex production conditions. In view of the above defects, in the embodiments of the present invention, a method and device for handling ton bags in automated production, a system for handling ton bags in automated production, a computer-readable storage medium, a processor, and a computer program product are provided.

[0032] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0033] The method embodiments provided in the embodiments of the present invention can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 1 is a hardware structure block diagram of a mobile terminal for a method for processing ton bags in automated production according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0034] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for handling ton bags in automated production in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof. The transmission device 106 is used to receive or send data via a network. The above network specific examples may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0035] According to an embodiment of the present invention, a method embodiment of a method for handling ton bags in automated production is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0036] Figure 2 : is a flow chart of a method for processing ton bags in automated production according to an embodiment of the present invention. Figure 2 As shown, the method comprises the following steps:

[0037] Step S202, when receiving a ton bag grabbing instruction, the ton bag image of the target area is collected by the image collection component, wherein the target area is the area where the ton bags are placed.

[0038] In this embodiment, when a ton bag grabbing instruction is received, a high-definition camera (i.e., image acquisition component) can be used to capture ton bag images of the area where the ton bags are placed, so that the position and posture information of the ton bags can be accurately determined, thereby providing important information for subsequent ton bag grabbing tasks.

[0039] Step S204, inputting the ton bag image into the ton bag recognition model, so as to process the ton bag image using the ton bag recognition model to obtain the target position information of the ton bag ears of the target ton bag corresponding to the ton bag image, wherein the ton bag recognition model is a model trained by machine learning using multiple sets of training data, and each set of the multiple sets of training data includes: a sample ton bag image and the position information of the sample ton bag ears corresponding to the sample ton bag image.

[0040] In this embodiment, the ton bag image can be input into a ton bag recognition model obtained by machine learning training using multiple sets of training data, and the ton bag image can be processed to obtain the target position information of the ton bag ears of the target ton bag corresponding to the ton bag image. The ton bag ear position information in the ton bag image can be accurately identified, thereby realizing intelligent ton bag processing operations and improving efficiency.

[0041] Step S206, generating a ton bag grabbing instruction according to the target location information.

[0042] In this embodiment, a ton bag grabbing instruction can be generated according to the target position information, thereby realizing automatic control of the operating equipment, ensuring that the equipment accurately grabs the target equipment, and reducing manual intervention and operation time.

[0043] Step S208, sending the ton bag grabbing instruction to the industrial computer, so as to use the industrial computer to control the industrial robot to grab the target ton bag according to the ton bag grabbing instruction and place the target ton bag at the designated location.

[0044] In this embodiment, by sending a ton bag grabbing instruction to an industrial computer, using the industrial computer to control an industrial robot to perform ton bag grabbing operations and place the target ton bag at a designated location, automatic grabbing and placing operations of ton bags can be achieved.

[0045] Figure 3 is a flow chart of a method for processing ton bags in optional automated production according to an embodiment of the present invention, such as Figure 3 As shown, after the transport vehicle is parked in place as required, the vehicle is scanned and the position of the ton bag is located using vision. The bag is then moved and placed according to the instructions of the system.

[0046] As can be seen from the above, in an embodiment of the present invention, when a ton bag grabbing instruction is received, a ton bag image of a target area is collected by an image acquisition component, wherein the target area is an area where ton bags are placed; the ton bag image is input into a ton bag recognition model, and the ton bag image is processed by the ton bag recognition model to obtain the target position information of the ton bag ears of the target ton bag corresponding to the ton bag image, wherein the ton bag recognition model is a model obtained by machine learning training using multiple sets of training data, and each of the multiple sets of training data includes: a sample ton bag image and position information of the sample ton bag ears corresponding to the sample ton bag image; a ton bag grabbing instruction is generated according to the target position information; the ton bag grabbing instruction is sent to an industrial computer, so that the industrial computer is used to control the industrial robot to grab the target ton bag according to the ton bag grabbing instruction, and place the target ton bag at a designated position, thereby achieving the purpose of utilizing and improving the YOLOv5 algorithm, identifying the ton bag through a high-definition camera, and then using a robotic arm to grab and unload, thereby achieving the technical effect of efficiently and accurately identifying and grabbing ton bags, and improving the flexibility and adaptability of production.

[0047] The technical solution provided by the embodiment of the present invention solves the technical problem that the grasping robots used in industrial automation production in the related technology have great limitations and cannot adapt to relatively complex production conditions.

[0048] According to the above-mentioned embodiment of the present invention, when a ton bag grabbing instruction is received, the ton bag image of the target area is collected by the image acquisition component, including: when the sensor arranged at the image acquisition component detects that the target ton bag is transported to the area where the image acquisition component is located, a ton bag grabbing instruction is generated; and the ton bag grabbing instruction is sent to the image acquisition component, so as to use the image acquisition component to collect the ton bag image of the target area.

[0049] In this embodiment, when a sensor provided at the high-definition camera (i.e., image acquisition component) detects that a target ton bag is transported to the area where the high-definition camera is located, a ton bag grabbing instruction is generated and sent to the high-definition camera so that the high-definition camera can be used to capture the ton bag image of the target area.

[0050] According to the above embodiment of the present invention, before the ton bag image is input into the ton bag recognition model to process the ton bag image using the ton bag recognition model to obtain the target position information of the ton bag ears of the target ton bag corresponding to the ton bag image, the method for processing ton bags in automated production also includes: obtaining a plurality of sample ton bag images and position information of a plurality of sample ton bag ears corresponding to the plurality of sample ton bag images; training a plurality of groups of training data including a plurality of sample ton bag images and position information of a plurality of sample ton bag ears to obtain a ton bag recognition model.

[0051] In this embodiment, multiple samples can be acquired to obtain the position information of the bag ears of multiple sample ton bags corresponding to the image, and the acquired multiple sets of training data are trained to obtain a ton bag recognition model. By acquiring multiple sets of training data and training the ton bag recognition model, accurate recognition of the positions of the bag ears of the ton bag in the ton bag image can be achieved.

[0052] According to the above embodiment of the present invention, multiple groups of training data including multiple sample ton bag images and multiple sample ton bag ear position information are trained to obtain a ton bag recognition model, including: acquiring multiple historical ton bag images acquired by an image acquisition device at different angles and under different lighting conditions; preprocessing the multiple historical ton bag images to obtain multiple sample ton bag images, wherein the preprocessing includes: scaling processing, translation processing, rotation processing and radiation change processing; annotating the multiple sample ton bag images to obtain the multiple sample ton bag ear position information.

[0053] In this embodiment, a high-definition camera can be used to photograph the ears of ton bags, and one frame of the picture can be captured as a basic data set output, that is, a high-definition camera is used to collect a basic data set (that is, multiple historical ton bag images) at different angles and under different lighting conditions, and after filtering and denoising the basic data set, the data set is expanded using scaling, translation, rotation, radial transformation and other methods to enhance the adaptability of ton bag recognition, and then the label img tool (that is, image annotation tool) is used to annotate the expanded data set to create an image data set to obtain multiple sample ton bag images.

[0054] Figure 4 is a general structural diagram of a method for processing ton bags in automated production according to an embodiment of the present invention, such as Figure 4 As shown, photos of ton bags at different angles and under different lighting conditions (i.e., data sets) are obtained, and the data sets are scaled, translated, rotated, and radially transformed to expand the data sets. Image annotation software is used to annotate the gripping parts of the ton bags and a self-made ton bag data set is made.

[0055] It should be noted that the production of the above dataset consists of two parts: basic dataset and dataset enhancement.

[0056] Figure 5 is a general structural diagram of a method for processing ton bags in automated production according to an embodiment of the present invention, such as Figure 5 As shown in the figure, the production of the above basic data set refers to using the video shot in the loading and unloading workshop to generate a large number of pictures with ton bag handles frame by frame, and using label img as the basis to annotate the data set.

[0057] It should be noted that the production of the above-mentioned basic data set needs to include different lighting environments and ton bag handles at different angles to simulate the recognition status of the target when the machine is in working condition. The above-mentioned different environments mainly include dark indoor environments and strong outdoor light environments, and the above-mentioned different angles are various deflection angles.

[0058] Figure 6 is a data annotation schematic diagram of a method for processing ton bags in automated production according to an embodiment of the present invention, and the data set enhancement part includes: Figure 6 The extended scaling, translation, rotation, and affine transformation shown are used to increase the data volume of the dataset, enrich the image types of the dataset, make it closer to the actual shooting situation, and enhance the adaptability of ton bag recognition.

[0059] According to the above embodiment of the present invention, multiple groups of training data including multiple sample ton bag images and multiple sample ton bag ears' position information are trained to obtain a ton bag recognition model, including: determining that the basic network model of the ton bag recognition model is the target detection model YOLOv5; using multiple groups of training data including multiple sample ton bag images and multiple sample ton bag ears' position information to train the target detection model YOLOv5 to obtain the ton bag recognition model.

[0060] It should be noted that the above-mentioned target detection model is the current mainstream single-stage target detection model, which has the characteristics of high performance, light weight, fast running speed, and high recognition accuracy. It is widely used in the field of computer vision and meets the needs of ton bag detection.

[0061] In this embodiment, the basic network model of the ton bag recognition model is determined to be the target detection model YOLOv5, a YOLOv5 network model for ton bag ear recognition is built, and the target detection model YOLOv5 is trained and tested using the above image data set to obtain a trained ton bag ear detection model.

[0062] It should be noted that the above YOLOv5 algorithm is a target detection algorithm based on the You Only Look Once (YOLO) series, which has the characteristics of fast speed, high accuracy and stable algorithm.

[0063] Optionally, the above YOLOv5 algorithm may include the following key components: network structure, target box regression, positive and negative sample matching, and loss calculation.

[0064] It should be noted that the above target frame regression calculation formula includes b x 、b y 、b w and b hFour major formulas; the above positive and negative sample matching includes anchor prior box screening, Ground, anchor ratio calculation and division of positive and negative samples; the above loss calculation includes grid prediction, information comparison and confidence analysis, among which grid prediction is the prediction of each grid on the feature map, information comparison is the comparison between predicted information and real information, and confidence analysis indicates the reliability of the predicted box, and the larger the value, the more reliable it is.

[0065] The network structure of the above-mentioned ton bag recognition model includes an input end (Input), a backbone network (Backbone), a feature fusion network (Neck) and an output end (Output), wherein the backbone network (backbone) includes a CONV module, a CSPDarkNet53 and a SPPF module; the CONV module includes a convolution (Conv2d), a Batch Normalization and an activation function; CSPNetNet53 ensures that the amount of calculation is reduced and the reasoning speed is improved without reducing the accuracy of model detection and recognition; SPPF replaces the original SPP module; Neck draws on FPN and PANet, FPN is a feature pyramid, and PANet introduces a bottom-up feature fusion based on FPN.

[0066] like Figure 4 As shown in the figure, after the ton bag data set is prepared, it is determined to use the improved YOLOv5 detection network to build a ton bag detection model, and the network is trained, verified and tested using the above data set, and then the coordinates of the center point of the ton bag's gripper are output as the gripping point (i.e., the target position information).

[0067] In the embodiment of the present invention, the target frame regression formula is as follows: x =2σ(t x )-0.5+c x , b y =2σ(t y )-0.5+c y , b w =p w ×(2σ(t w )) 2 , b h =p h ×(2σ(t h )) 2 , where b x 、b y 、b w 、b h Respectively represent the x, y coordinates, width and height of the center point of the prediction box, c x 、c y They are the coordinates of the upper left corner of the network where the center of the prediction box is located, tx ,t y They represent the offset of the center point of the prediction box relative to the coordinates of the upper left corner of the network, t w ,t h Indicates the scaling ratio of the width and height of the prediction box relative to the width and height of the anchor, p w 、p h Indicates the width and height of the prior box anchor

[0068] Furthermore, the positive and negative sample matching is based on 9 different anchor aspect ratios, and the maximum value is taken as the ratio of GroundTruth and anchor, and the resulting prediction box is divided into positive and negative samples.

[0069] In an embodiment of the present invention, the loss calculation includes predicting the grid on the feature map and comparing the obtained prediction information with the real information. The larger the confidence value is, the more reliable the prediction box is.

[0070] In an embodiment of the present invention, the recognition accuracy and grasping success rate can be tested through the visual system, and the training effect can be checked through the verification set, wherein the verification set is related to the confidence, and the confidence is related to the accuracy and recall rate. The greater the confidence, the more accurate the category detection, and the smaller the confidence, the more comprehensive the image detection, which improves the accuracy of image recognition.

[0071] Optionally, the above tests may include: recognition accuracy test, capture success rate test and recognition rate analysis.

[0072] The recognition accuracy and grasping success rate are tested through the visual system, and the training effect is checked through the verification set: the verification set is related to the confidence level, which is related to the accuracy and recall rate. The greater the confidence level, the more accurate the category detection is, and the smaller the confidence level, the more comprehensive the image detection is, which improves the accuracy of image recognition.

[0073] In addition, after determining that the network detection model of the ton bag ears is the YOLOv5 target detection model, it is necessary to deploy the YOLOv5 network for ton bag detection on the industrial computer, install the high-definition camera above the ton bag ears, and shoot the ton bag ears in real time. After the industrial computer triggers the camera to capture a frame of the picture, the coordinate position of the target point is identified through the deep learning network and output to the industrial computer, which then grabs the ears of the ton bag.

[0074] The specific steps of the embodiment of the present invention are as follows: 1) image acquisition; 2) data set expansion; 3) production of ton bag image data set; 4) construction of network model; 5) ton bag ear image detection. Specifically, the ton bag ear detection model trained in the steps is deployed on an industrial computer, and the target points of the ton bag ears are detected and output to the industrial computer in real time. The industrial computer issues a command to grab the ton bag.

[0075] Through the technical solution provided by the above-mentioned embodiment of the present invention, for the scenario of unmanned unloading of ton bag transport vehicles, the YOLOv5 algorithm is utilized and improved to provide a ton bag recognition and grabbing method based on the improved YOLOv5, which aims to recognize ton bags in industrial scenarios and realize automatic grabbing. By improving the YOLOv5 model, it can accurately detect ton bags and grab them through a robotic arm or other device. At the same time, the ton bags are identified through a high-definition camera, and then the robotic arm is used to grab and unload. Through reasonable model improvement, sensor integration, mechanical control and algorithm optimization, efficient and accurate ton bag recognition and grabbing operations can be achieved in industrial scenarios, thereby improving production efficiency and reducing labor costs.

[0076] That is, the above technical solution provided by the embodiment of the present invention comprehensively considers various factors and effectively overcomes the technical problems that the existing target detection network based on deep learning is usually difficult to deploy on the edge computing platform due to the large number of parameters and model volume, and the detection accuracy of the small-volume network is greatly reduced, making it difficult to meet the requirements of practical applications. It mainly uses a lightweight target detection network based on improved YOLOv5 to achieve a network with high detection accuracy while having an extremely small volume. Through the deep learning method, the center point coordinates of the bag ears are accurately proposed and then output to the industrial computer for grabbing, which can improve the loading and unloading efficiency of ton bags and the safety of the loading and unloading process.

[0077] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0078] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0079] According to an embodiment of the present invention, there is also provided a device for processing ton bags in automated production for implementing the above-mentioned method for processing ton bags in automated production. Figure 7 Schematic diagram of a processing device for ton bags in automated production according to an embodiment of the present invention. Figure 7 As shown, the device includes: a collection module 701, a first processing module 703, a generation module 705 and a second processing module 707. The risk analysis device in the operation process of the power grid is described below.

[0080] The acquisition module 701 is used to acquire the image of the ton bag in the target area through the image acquisition component when receiving the ton bag grabbing instruction, wherein the target area is the area where the ton bag is placed.

[0081] The first processing module 703 is used to input the ton bag image into the ton bag recognition model so as to process the ton bag image using the ton bag recognition model to obtain the target position information of the ton bag ears of the target ton bag corresponding to the ton bag image, wherein the ton bag recognition model is a model trained by machine learning using multiple sets of training data, and each of the multiple sets of training data includes: a sample ton bag image and the position information of the sample ton bag ears corresponding to the sample ton bag image.

[0082] The generating module 705 is used to generate a ton bag grabbing instruction according to the target position information.

[0083] The second processing module 707 is used to send the ton bag grabbing instruction to the industrial computer, so as to use the industrial computer to control the industrial robot to grab the target ton bag according to the ton bag grabbing instruction and place the target ton bag at a designated location.

[0084] It should be noted here that the above-mentioned acquisition module 701, the first processing module 703, the generation module 705 and the second processing module 707 correspond to steps S202 to S208 in the above-mentioned embodiments, and the four modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the contents disclosed in the above-mentioned embodiments.

[0085] As can be seen from the above, in the scheme recorded in the above embodiment of the present invention, firstly, the acquisition module can be used to acquire the image of the ton bag in the target area through the image acquisition component when receiving the ton bag grabbing instruction, wherein the target area is the area where the ton bags are placed; secondly, the first processing module can be used to input the ton bag image into the ton bag recognition model, so as to process the ton bag image using the ton bag recognition model to obtain the target position information of the ton bag ears of the target ton bag corresponding to the ton bag image, wherein the ton bag recognition model is a model obtained by machine learning training using multiple sets of training data, and each of the multiple sets of training data includes: sample ton bags The image and the position information of the sample ton bag ears corresponding to the sample ton bag image; then the generation module can be used to generate the ton bag grabbing instruction according to the target position information; finally, the second processing module can be used to send the ton bag grabbing instruction to the industrial computer, so as to use the industrial computer to control the industrial robot to grab the target ton bag according to the ton bag grabbing instruction, and place the target ton bag at the designated position, thereby achieving the purpose of utilizing and improving the YOLOv5 algorithm, identifying the ton bag through a high-definition camera, and then using a robotic arm to grab and unload, thereby achieving the technical effect of efficiently and accurately identifying and grabbing ton bags, and improving the flexibility and adaptability of production.

[0086] The technical solution provided by the embodiment of the present invention solves the technical problem that the grasping robots used in industrial automation production in the related technology have great limitations and cannot adapt to relatively complex production conditions.

[0087] In an optional embodiment, the acquisition module includes: a generating unit, which is used to generate a ton bag grabbing instruction when a sensor arranged at the image acquisition component detects that a target ton bag is transported to an area where the image acquisition component is located; and a sending unit, which is used to send the ton bag grabbing instruction to the image acquisition component, so as to use the image acquisition component to acquire an image of the ton bag in the target area.

[0088] In an optional embodiment, the device for processing ton bags in the automated production also includes: an acquisition module, which is used to obtain a plurality of sample ton bag images and position information of a plurality of sample ton bag ears corresponding to the plurality of sample ton bag images before inputting the ton bag image into the ton bag recognition model to process the ton bag image using the ton bag recognition model to obtain the target position information of the ton bag ears of the target ton bag corresponding to the ton bag image; and a training module, which is used to train a plurality of groups of training data including a plurality of sample ton bag images and position information of a plurality of sample ton bag ears to obtain the ton bag recognition model.

[0089] In an optional embodiment, the training module includes: an acquisition unit, used to acquire multiple historical ton bag images acquired by an image acquisition device at different angles and different lighting conditions; a preprocessing unit, used to preprocess the multiple historical ton bag images to obtain multiple sample ton bag images, wherein the preprocessing includes: scaling processing, translation processing, rotation processing and radiation change processing; a labeling unit, used to label the multiple sample ton bag images to obtain the location information of the bag ears of the multiple sample ton bags.

[0090] In an optional embodiment, the training module includes: a determination unit, used to determine that the basic network model of the ton bag recognition model is the target detection model YOLOv5; a training unit, used to train the target detection model YOLOv5 using multiple groups of training data including multiple sample ton bag images and multiple sample ton bag ear position information to obtain the ton bag recognition model.

[0091] According to another aspect of an embodiment of the present invention, a system for processing ton bags in automated production is provided. The system for processing ton bags in automated production uses any of the above methods for processing ton bags in automated production.

[0092] According to another aspect of an embodiment of the present invention, a processor is further provided, and the processor is used to run a program, wherein when the program is run, any one of the above-mentioned methods for processing ton bags in automated production is executed.

[0093] According to another aspect of an embodiment of the present invention, there is also provided a computer program product, comprising computer instructions, which, when executed by a processor, execute any one of the above methods for processing ton bags in automated production.

[0094] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium comprising a stored program, wherein the program executes any one of the above-mentioned methods for processing ton bags in automated production.

[0095] Optionally, in this embodiment, the computer-readable storage medium may be located in any one of the computer terminals in a computer terminal group in a computer network, or in any one of the communication devices in a communication device group.

[0096] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for executing the following steps: when a ton bag grabbing instruction is received, a ton bag image of a target area is captured by an image acquisition component, wherein the target area is an area where ton bags are placed; the ton bag image is input into a ton bag recognition model, and the ton bag image is processed using the ton bag recognition model to obtain target position information of ton bag ears of a target ton bag corresponding to the ton bag image, wherein the ton bag recognition model is a model trained by machine learning using multiple sets of training data, and each of the multiple sets of training data includes: a sample ton bag image and position information of a sample ton bag ears corresponding to the sample ton bag image; a ton bag grabbing instruction is generated according to the target position information; the ton bag grabbing instruction is sent to an industrial computer, so that the industrial computer is used to control an industrial robot to grab the target ton bag according to the ton bag grabbing instruction, and place the target ton bag at a designated position.

[0097] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for executing the following steps: generating a ton bag grabbing instruction when a sensor provided at the image acquisition component detects that a target ton bag is transported to the area where the image acquisition component is located; and sending the ton bag grabbing instruction to the image acquisition component to use the image acquisition component to capture an image of the ton bag in the target area.

[0098] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for executing the following steps: obtaining location information of multiple sample big bag images and multiple sample big bag ears corresponding to the multiple sample big bag images; training multiple groups of training data including multiple sample big bag images and multiple sample big bag ears to obtain a big bag recognition model.

[0099] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for executing the following steps: acquiring multiple historical ton bag images acquired by an image acquisition device at different angles and under different lighting conditions; preprocessing the multiple historical ton bag images to obtain multiple sample ton bag images, wherein the preprocessing includes: scaling processing, translation processing, rotation processing, and radiation change processing; annotating the multiple sample ton bag images to obtain the position information of the bag ears of multiple sample ton bags.

[0100] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for performing the following steps: determining that the basic network model of the ton bag recognition model is the target detection model YOLOv5; training the target detection model YOLOv5 using multiple sets of training data including multiple sample ton bag images and multiple sample ton bag ear position information to obtain the ton bag recognition model.

[0101] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0102] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0103] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0104] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0105] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0106] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0107] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for handling ton bags in automated production. It is characterized in that include: When receiving a ton bag grabbing instruction, the ton bag image of the target area is collected by the image acquisition component, wherein the target area is the area where the ton bags are placed; The ton bag image is input into a ton bag recognition model, so as to process the ton bag image using the ton bag recognition model to obtain target position information of ton bag ears of a target ton bag corresponding to the ton bag image, wherein the ton bag recognition model is a model trained by machine learning using multiple sets of training data, each of the multiple sets of training data includes: a sample ton bag image and position information of a sample ton bag ear corresponding to the sample ton bag image; Generate a ton bag grabbing instruction according to the target position information; The ton bag grabbing instruction is sent to an industrial computer, so that the industrial computer controls an industrial robot to grab the target ton bag according to the ton bag grabbing instruction and place the target ton bag at a designated location.

2. The method for processing ton bags in automated production according to claim 1, It is characterized in that When receiving the instruction to grab the ton bag, the image of the ton bag in the target area is collected by the image acquisition component, including: When the sensor arranged at the image acquisition component detects that the target ton bag is transported to the area where the image acquisition component is located, the ton bag grabbing instruction is generated; The ton bag grabbing instruction is sent to the image acquisition component, so that the ton bag image of the target area is acquired by the image acquisition component.

3. The method for processing ton bags in automated production according to claim 1, It is characterized in that Before inputting the ton bag image into the ton bag recognition model to process the ton bag image using the ton bag recognition model to obtain target position information of the ton bag ears of the target ton bag corresponding to the ton bag image, the method further includes: Acquire a plurality of sample big bag images and position information of a plurality of sample big bag ears corresponding to the plurality of sample big bag images; The multiple sets of training data including multiple sample ton bag images and multiple positions of the bag ears of the sample ton bag are trained to obtain the ton bag recognition model.

4. The method for processing ton bags in automated production according to claim 3, It is characterized in that Training a plurality of sets of the training data including a plurality of the sample ton bag images and a plurality of the position information of the bag ears of the sample ton bag to obtain the ton bag recognition model includes: Acquire multiple historical ton bag images collected by image acquisition equipment at different angles and under different lighting conditions; Preprocessing the plurality of historical ton bag images to obtain the plurality of sample ton bag images, wherein the preprocessing includes: scaling processing, translation processing, rotation processing and radiation change processing; The plurality of sample ton bag images are labeled to obtain the position information of the bag ears of the plurality of sample ton bags.

5. The method for processing ton bags in automated production according to claim 3, It is characterized in that Training a plurality of sets of the training data including a plurality of the sample ton bag images and a plurality of the position information of the bag ears of the sample ton bag to obtain the ton bag recognition model includes: Determine that the basic network model of the ton bag recognition model is the target detection model YOLOv5; The target detection model YOLOv5 is trained using multiple groups of the training data including multiple sample ton bag images and multiple positions of the bag ears of the sample ton bag to obtain the ton bag recognition model.

6. A processing device for ton bags in automated production, It is characterized in that include: A collection module, used for collecting the image of the ton bag in the target area through the image collection component when receiving the ton bag grabbing instruction, wherein the target area is the area where the ton bag is placed; A first processing module is used to input the ton bag image into a ton bag recognition model, so as to process the ton bag image using the ton bag recognition model, so as to obtain target position information of ton bag ears of a target ton bag corresponding to the ton bag image, wherein the ton bag recognition model is a model obtained by machine learning training using multiple sets of training data, and each of the multiple sets of training data includes: a sample ton bag image and position information of a sample ton bag ear corresponding to the sample ton bag image; A generating module, used for generating a ton bag grabbing instruction according to the target position information; The second processing module is used to send the ton bag grabbing instruction to the industrial computer, so as to use the industrial computer to control the industrial robot to grab the target ton bag according to the ton bag grabbing instruction and place the target ton bag at a designated location.

7. A system for handling ton bags in automated production. It is characterized in that The system for handling bulk bags in automated production uses the method for handling bulk bags in automated production as described in any one of claims 1 to 5 above.

8. A computer-readable storage medium, It is characterized in that The computer-readable storage medium includes a stored program, wherein the program executes the method for handling ton bags in automated production according to any one of claims 1 to 5.

9. A processor, It is characterized in that The processor is used to run a program, wherein the program, when running, executes the method for processing ton bags in automated production as described in any one of claims 1 to 5.

10. A computer program product comprising computer instructions, It is characterized in that When the computer instructions are executed by the processor, the method for processing ton bags in automated production described in any one of claims 1 to 5 is performed.

Images