A Load Rate Calculation Method Based on Image Recognition

Abstract

This invention discloses a loading rate calculation method based on image recognition, comprising the following steps: (1) acquiring multiple videos of different types of trucks loading goods, and splitting them into sample images as training sample images; (2) obtaining the annotation information of the training sample images and saving it as an annotation file; (3) inputting the training sample images and the annotation file into the YOLO-V3 model for training to obtain a cargo and cargo compartment rear contour recognition model; (4) acquiring multiple real-time captured images of the truck loading scene, and inputting the real-time images into the cargo and cargo compartment rear contour recognition model for recognition, outputting the loaded cargo detection box and the cargo compartment rear contour detection box in the real-time images; (5) calculating the actual distance L between the loaded cargo detection box and the cargo compartment rear contour detection box; (6) obtaining the actual cargo compartment length K of each truck and calculating the truck loading rate. This invention improves the efficiency of truck loading rate recognition.

Description

Technical Field

[0001] This invention relates to the technical field of particleboard surface defect acquisition, and more specifically, to a method for calculating loading rate based on image recognition. Background Technology

[0002] With the increasing popularity of online consumption, the domestic logistics and transportation industry has experienced rapid development. Among them, road freight has become the most important mode of logistics transportation due to its advantages such as fast transportation speed, high reliability, high mobility, and good economic benefits. During the transportation of goods by truck, the loading rate of the cargo compartment directly affects truck scheduling and unit freight costs, making it a key indicator determining the profitability of the entire logistics industry. Currently, most logistics units rely on manual evaluation methods, which are difficult to guarantee in terms of both statistical accuracy and reliability. Furthermore, they cannot obtain real-time statistical data, hindering overall scheduling and operation to achieve cost reduction and efficiency improvement. Only by utilizing technological means can long-term, efficient management of the transportation process be achieved. The discrepancy between the calculated loading rate of the truck compartment and the actual loading rate makes reliability difficult to guarantee; existing technologies struggle to ensure accuracy and real-time performance, resulting in limited improvement in overall operational efficiency. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a loading rate calculation method based on image recognition. By using image recognition to measure distances within the carriage and combining this with multi-point calculation of the average distance of the remaining space, the loading rate of the carriage can be finally calculated.

[0004] A method for calculating load rate based on image recognition includes the following steps:

[0005] (1) Collect multiple videos of different models of trucks loading goods, and split them into frames to form sample images as training sample images; the multiple videos include the area of ​​the truck loading goods and the outline of the rear of the truck body.

[0006] (2) Label the area of ​​the truck loaded with goods and the outline of the rear of the truck in the training sample image respectively, and save the labeling information of the training sample image as a labeling file;

[0007] (3) Input the training sample images and annotation files into the YOLO-V3 model for training to obtain the cargo and carriage rear contour recognition model; the cargo and carriage rear contour recognition model is used to input images and output the loaded cargo detection box and carriage rear contour detection box of the images.

[0008] (4) Acquire multiple real-time captured images of the truck loading scene, and input the images into the cargo and cargo compartment rear end contour recognition model for recognition, and output the loaded cargo detection box and cargo compartment rear end contour detection box in the images; the images include the area of ​​the truck loading cargo and the contour of the cargo compartment rear end.

[0009] (5) Calculate the actual distance L between the loaded cargo detection frame and the rear profile detection frame of the carriage using the loaded cargo detection frame and the carriage rear profile detection frame from step (4);

[0010] (6) Obtain the actual cargo length K of each truck by the actual distance L between the loaded cargo detection frame and the cargo tail contour detection frame in step (5), and calculate the truck loading rate as (KL) / K * 100%.

[0011] Specifically, the annotation file is an XML file.

[0012] Specifically, the YOLO-v3 model includes the Darknet-53 network.

[0013] Specifically, the method for calculating the distance L between the loaded cargo detection frame and the rear end contour detection frame of the carriage in step (5) is as follows:

[0014] S1. Calculate the coordinates of the center point on the left side of the loaded goods detection frame as the coordinates of the center point on the left side of the loaded goods detection frame, and the coordinates of the center point on the right side as the coordinates of the center point on the left side of the loaded goods detection frame;

[0015] S2. Calculate the center point coordinates of the left side as the center point coordinates of the left side of the carriage rear contour detection frame and the center point coordinates of the right side as the center point coordinates of the right side of the carriage rear contour detection frame, based on the carriage rear contour detection frame.

[0016] S3. Calculate the pixel distance L1 from the center point coordinates of the left carriage rear contour detection box to the left center point coordinates of the loaded cargo detection box, and calculate the pixel distance L2 from the center point coordinates of the right carriage rear contour detection box to the right center point coordinates of the loaded cargo detection box.

[0017] S4. By combining the pre-collected pixel distance between the center point of the cargo frame and the center point of the rear contour of the carriage with the actual measured distance between the center point of the cargo edge and the center point of the rear contour of the carriage, a relationship model between pixel distance and actual distance is fitted, which can realize the conversion from pixel distance to actual distance.

[0018] S5. Based on the relationship model fitted in step S4, convert the pixel distance L1 and pixel distance L2 in step S3 into the corresponding actual distances L1' and L2', respectively, and calculate the average distance L between the actual distances L1' and L2' as the distance L between the loaded cargo detection frame and the rear contour detection frame of the carriage.

[0019] A system for calculating load rate based on image recognition using the above method includes:

[0020] The acquisition module is used to collect multiple videos of different types of trucks loading goods; the acquisition module is also used to split the multiple videos into frames and process them into sample images as training sample images; the multiple videos can capture the area of ​​the truck loading goods and the outline position of the rear end of the truck body;

[0021] The annotation module is used to annotate the area of ​​the truck loaded with goods and the outline of the rear of the truck's cargo compartment in the training sample images, respectively, and save the annotation information of the training sample images as an annotation file.

[0022] The training module is used to input training sample images and annotation files into the YOLO-V3 model for training, and to obtain a cargo and carriage rear contour recognition model; the cargo and carriage rear contour recognition model is used to input images and can output the loaded cargo detection box and the carriage rear contour detection box of the image.

[0023] The recognition module is used to acquire multiple real-time captured images of a truck loading scene, and input the images into the cargo and rear end contour recognition model for recognition, and output the loaded cargo detection box and the rear end contour detection box in the images; the images include the area of ​​the truck loaded with cargo and the contour of the rear end of the truck.

[0024] The distance conversion module is used to calculate the actual distance L between the loaded cargo detection frame and the rear end contour detection frame of the carriage based on the loaded cargo detection frame and the carriage rear end contour detection frame obtained by the recognition module.

[0025] The loading rate calculation module is used to obtain the actual cargo length K of each truck by using the actual distance L between the loaded cargo detection frame and the cargo box rear contour detection frame of the distance conversion module, and to calculate the truck loading rate as (KL) / K * 100%.

[0026] The acquisition module, annotation module, training module, recognition module, distance conversion module, and load rate calculation module are connected in sequence.

[0027] Specifically, the acquisition module is an IP camera installed at the top of the rear of the truck bed or a camera fixed to the steering wheel to capture video of the truck loading scene.

[0028] Specifically, the actual distance L between the loaded cargo detection frame and the rear contour detection frame of the carriage obtained by the distance conversion module is transmitted to the loading rate calculation module in JSON format.

[0029] Specifically, the acquisition module also includes several infrared ranging sensors; the infrared ranging sensors are deployed along the longitudinal direction of the truck body at the front and rear edges of the truck body.

[0030] Specifically, the annotation file is an XML file.

[0031] Specifically, the YOLO-v3 model includes the Darknet-53 network.

[0032] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0033] This invention first acquires and annotates images of truck loading scenes to obtain annotation files. Then, the images and annotation files are input together into a pre-built YOLO-V3 model using Darknet-53 as the feature extraction network. The model is trained using the images and annotation files to obtain a cargo region recognition model and a truck bed rear contour recognition model. By borrowing the design of Darknet-19 and ResNet from YOLOv2 and integrating the preceding FPN to form Darknet-53 convolutional layers, a fast object detection algorithm is formed, thereby improving the efficiency of truck loading rate recognition. In the application of truck bed loading rate statistics, this invention can accurately identify and calculate the loading rate, record it in the system in a timely manner, and identify and alert trucks that do not meet the standards. Through overall scheduling and operation, it largely avoids the waste of logistics resources and reduces logistics costs. Attached Figure Description

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 This is a flowchart illustrating a method according to an embodiment of the invention.

[0036] Figure 2 This is a schematic diagram of the internal structure of an embodiment of the invention.

[0037] Figure 3 This is a system principle block diagram of an embodiment of the invention.

[0038] In the diagram: 1. Left outline; 2. Right outline; 3. Loaded cargo detection frame; 4. Center point coordinates of the left rear end outline detection frame; 5. Center point coordinates of the right rear end outline detection frame; 6. Center point coordinates of the left side of the loaded cargo detection frame; 7. Center point coordinates of the right side of the loaded cargo detection frame; 10. Left outline frame; 11. Right outline frame; 12. Truck loaded with cargo. Detailed Implementation

[0039] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby providing a clearer and more explicit definition of the scope of protection of the present invention.

[0040] It should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of the invention is in use. They are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. In addition, the terms "first," "second," and "third," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0041] Furthermore, terms such as "horizontal," "vertical," and "sag" do not imply that components must be absolutely horizontal or suspended, but rather that they can be slightly tilted. For example, "horizontal" simply means that its direction is more horizontal relative to "vertical," and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0042] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0043] like Figure 1 , Figure 2 As shown, this embodiment of a method for calculating load rate based on image recognition includes the following steps:

[0044] (1) Collect multiple videos of different models of trucks loading goods 12, and split them into frames to create sample images for training. The multiple videos include the area of ​​the truck loading goods 12 and the outline of the rear of the truck bed. In this embodiment of the invention, the specific method for collecting multiple videos of different models of trucks loading goods 12 is as follows: First, a video of the truck loading scene is captured by an IP camera (a wide-angle camera is required) or a fixed camera at the helm, which can capture the area of ​​the truck loading goods 12 and the outline of the rear of the truck bed. Frames are extracted at intervals to generate images, which are then used as training sample images. The outline of the rear of the truck bed includes the left outline 1 and the right outline 2.

[0045] (2) Label the area of ​​the truck carrying goods 12 and the outline of the rear of the truck compartment in the training sample image respectively, and save the label information of the training sample image as a label file; the label file is an XML file. The training sample image generated above is labeled manually. The labelImg tool is used to select the cargo stacking area and the outline of the rear of the truck compartment on the left and right sides in the form of a rectangle, and generate a parsable XML label file.

[0046] (3) Input the training sample images and annotation files into the YOLO-v3 model for training to obtain the cargo and carriage rear contour recognition model. The cargo and carriage rear contour recognition model is used to input images and output the loaded cargo detection boxes and carriage rear contour detection boxes. The YOLO-v3 model includes the Darknet-53 network. The YOLO-v3 model is an end-to-end object detection model. The basic idea of ​​the algorithm is: first, extract features from the input features through a feature extraction network to obtain a feature map of a specific size. The input image is divided into 13×13 grid cells. Then, if the center coordinates of an object in the real box fall into a grid cell, then the grid cell will predict the object. Each object has a fixed number of bounding boxes. The YOLO-v3 model has three bounding boxes, and logistic regression is used to determine the regression boxes used for prediction. The YOLO-v3 model in this embodiment uses a 53-layer convolutional network, which is composed of residual units stacked together. In terms of the balance between classification accuracy and efficiency. The Darknet-53 model used in this embodiment of the invention performs better than ResNet-101, ResNet-152 and Darknet-19.

[0047] (4) Acquire multiple real-time captured images of the truck loading scene, and input the images into the cargo and cargo compartment rear end contour recognition model for recognition. Output the loaded cargo detection box 3 and the cargo compartment rear end contour detection box in the real-time images. The cargo compartment rear end contour detection box includes the left contour box 10 and the right contour box 11, thus obtaining three rectangular boxes. The real-time images include the area of ​​the truck loading cargo 12 and the contour of the cargo compartment rear end.

[0048] (5) Calculate the actual distance L between the loaded cargo detection frame 3 and the rear end contour detection frame of the carriage using the loaded cargo detection frame 3 and the carriage rear end contour detection frame from step (4). The specific method is as follows:

[0049] S1. Calculate the center point coordinates of the left side of the loaded goods detection frame 3 as the center point coordinates of the left side of the loaded goods detection frame 6 and the center point coordinates of the right side as the center point coordinates of the left side of the loaded goods detection frame 6 respectively;

[0050] S2. Calculate the center point coordinates of the left side as the center point coordinates of the left side of the carriage rear end contour detection frame (4) and the center point coordinates of the right side as the center point coordinates of the right side of the carriage rear end contour detection frame (5) based on the center point coordinates of the left side of the carriage rear end contour detection frame.

[0051] S3. Calculate the pixel distance L1 from the center point coordinate 4 of the left carriage rear contour detection box to the left center point coordinate 6 of the loaded cargo detection box, and calculate the pixel distance L2 from the center point coordinate 5 of the right carriage rear contour detection box to the right center point coordinate 7 of the loaded cargo detection box.

[0052] S4. By combining the pre-collected pixel distance between the center point of the cargo frame and the center point of the rear contour of the carriage with the actual measured distance between the center point of the cargo edge and the center point of the rear contour of the carriage, a relationship model between pixel distance and actual distance is fitted, which can realize the conversion from pixel distance to actual distance.

[0053] S5. Based on the fitted relationship model in step S4, convert the pixel distances L1 and L2 in step S3 into corresponding actual distances L1' and L2', respectively, and calculate the average distance L between the actual distances L1' and L2' as the distance L between the loaded cargo detection frame 3 and the rear contour detection frame of the carriage. The output of this embodiment is in very small JSON format.

[0054] (6) Obtain the actual cargo box length K of each truck by using the actual distance L between the loaded cargo detection frame 3 and the cargo box tail contour detection frame in step (5): When the vehicle is registered, basic information such as cargo box length will be entered into the system carried by this embodiment of the invention. Different types of trucks have different cargo box lengths. The vehicle can be locked according to the unique ID returned by the IP camera, and then the cargo box length information K of the current vehicle can be obtained by association matching. Finally, the truck loading rate is calculated as (KL) / K * 100% based on the actual cargo box length K of each truck entered by the system.

[0055] like Figure 3 As shown, an embodiment of the present invention provides a system for calculating load rate based on image recognition using the above method, comprising:

[0056] The acquisition module is used to collect multiple videos of different models of trucks loading goods 12. The acquisition module also processes these multiple videos into frames to create sample images for use as training samples. The multiple videos can capture the area where the trucks are loading goods 12 and the outline of the rear of the truck bed. The acquisition module is an IP camera mounted on the top of the rear of the truck bed or a camera fixed to the steering wheel to capture videos of the truck loading scene.

[0057] The annotation module is used to annotate the area of ​​the truck loaded with goods 12 and the outline of the rear end of the truck's cargo compartment in the training sample image, respectively, and save the annotation information of the training sample image as an annotation file.

[0058] The training module is used to input training sample images and annotation files into the YOLOv3 model for training, resulting in a cargo and wagon rear contour recognition model. This model takes an image as input and outputs bounding boxes for loaded cargo and wagon rear contours.

[0059] The recognition module is used to acquire multiple on-site images of the truck loading scene captured in real time, and input the on-site images into the cargo and rear end contour recognition model for recognition, and output the loaded cargo detection box 3 and the rear end contour detection box of the truck in the on-site image;

[0060] The distance conversion module is used to calculate the actual distance L between the loaded cargo detection frame 3 and the rear contour detection frame of the carriage obtained by the recognition module. The actual distance L between the loaded cargo detection frame 3 and the rear contour detection frame of the carriage obtained by the distance conversion module is transmitted to the loading rate calculation module in JSON format and can be transmitted to the loading rate calculation module via 4G / 5G network.

[0061] The loading rate calculation module is used to obtain the actual cargo length K of each truck by using the actual distance L between the loaded cargo detection frame 3 of the distance conversion module and the cargo box rear contour detection frame, and to calculate the truck loading rate as (KL) / K * 100%.

[0062] The acquisition module, annotation module, training module, recognition module, distance conversion module, and load rate calculation module are connected in sequence.

[0063] The acquisition module in this embodiment also includes several infrared ranging sensors. These infrared ranging sensors are deployed along the longitudinal direction of the truck bed at the front and rear edges of the bed, and are used to correct the data collected from the ranging samples within the truck bed.

[0064] Although embodiments of the present invention have been described in conjunction with the accompanying drawings, the patent holder may make various modifications or alterations within the scope of the appended claims. As long as these modifications or alterations do not exceed the scope of protection described in the claims, they should all be within the scope of protection of the present invention. Specific examples have been used to illustrate the principles and embodiments of the present invention. These examples are merely for the purpose of helping to understand the method and core ideas of the present invention. The above are only preferred embodiments of the present invention. It should be noted that due to the limitations of written expression, and the objective existence of infinite specific structures, those skilled in the art can make several improvements, modifications, or variations without departing from the principles of the present invention, and can also combine the above technical features in an appropriate manner. These improvements, modifications, variations, or combinations, or the direct application of the inventive concept and technical solution to other situations without modification, should all be considered within the scope of protection of the present invention.

Claims

1. A method for calculating load rate based on image recognition, characterized in that: Includes the following steps: (1) Collect multiple videos of different models of trucks loading goods, and split them into frames to form sample images as training sample images; the multiple videos include the area of ​​the truck loading goods and the outline of the rear of the truck body. (2) Label the area of ​​the truck loaded with goods and the outline of the rear of the truck in the training sample image respectively, and save the labeling information of the training sample image as a labeling file; (3) Input the training sample images and annotation files into the YOLO-V3 model for training to obtain the cargo and carriage rear contour recognition model; the cargo and carriage rear contour recognition model is used to input images and output the loaded cargo detection box and carriage rear contour detection box of the images. (4) Acquire multiple real-time captured images of the truck loading scene, and input the images into the cargo and cargo compartment rear end contour recognition model for recognition, and output the loaded cargo detection box and cargo compartment rear end contour detection box in the images; the images include the area of ​​the truck loading cargo and the contour of the cargo compartment rear end. (5) Calculate the actual distance L between the loaded cargo detection frame and the rear profile detection frame of the carriage using the loaded cargo detection frame and the rear profile detection frame of the carriage from step (4); the specific method for calculating the distance L between the loaded cargo detection frame and the rear profile detection frame of the carriage in step (5) is as follows: S1. Calculate the center point coordinates of the left side of the loaded goods detection frame as the center point coordinates of the left side of the loaded goods detection frame and the center point coordinates of the right side as the center point coordinates of the right side of the loaded goods detection frame; S2. Calculate the center point coordinates of the left side as the center point coordinates of the left side of the carriage rear contour detection frame and the center point coordinates of the right side as the center point coordinates of the right side of the carriage rear contour detection frame, based on the carriage rear contour detection frame. S3. Calculate the pixel distance L1 from the center point coordinates of the left carriage rear contour detection box to the left center point coordinates of the loaded cargo detection box, and calculate the pixel distance L2 from the center point coordinates of the right carriage rear contour detection box to the right center point coordinates of the loaded cargo detection box. S4. By combining the pre-collected pixel distance between the center point of the cargo frame and the center point of the rear contour of the carriage with the actual measured distance between the center point of the cargo edge and the center point of the rear contour of the carriage, a relationship model between pixel distance and actual distance is fitted, which can realize the conversion from pixel distance to actual distance. S5. Based on the relationship model fitted in step S4, convert the pixel distance L1 and pixel distance L2 in step S3 into the corresponding actual distances L1' and L2' respectively, and calculate the average distance L between the actual distances L1' and L2' as the distance L between the loaded cargo detection frame and the rear contour detection frame of the carriage. (6) Obtain the actual cargo length K of each truck by the actual distance L between the loaded cargo detection frame and the cargo tail contour detection frame in step (5), and calculate the truck loading rate as (KL) / K * 100%.

2. The loading rate calculation method based on image recognition according to claim 1, characterized in that: The annotation file is an XML file.

3. The loading rate calculation method based on image recognition according to claim 1, characterized in that: The YOLOv3 model includes the Darknet-53 network.

4. A system for calculating load rate based on image recognition using any one of the methods of claims 1-3, comprising: The acquisition module is used to collect multiple videos of different types of trucks loading goods; the acquisition module is also used to split the multiple videos into frames and process them into sample images as training sample images; the multiple videos can capture the area of ​​the truck loading goods and the outline position of the rear end of the truck body; The annotation module is used to annotate the area of ​​the truck loaded with goods and the outline of the rear of the truck's cargo compartment in the training sample images, respectively, and save the annotation information of the training sample images as an annotation file. The training module is used to input training sample images and annotation files into the YOLO-V3 model for training, and to obtain a cargo and carriage rear contour recognition model; the cargo and carriage rear contour recognition model is used to input images and can output the loaded cargo detection box and the carriage rear contour detection box of the image. The recognition module is used to acquire multiple real-time captured images of a truck loading scene, and input the images into the cargo and rear end contour recognition model for recognition, and output the loaded cargo detection box and the rear end contour detection box in the images; the images include the area of ​​the truck loaded with cargo and the contour of the rear end of the truck. The distance conversion module is used to calculate the actual distance L between the loaded cargo detection frame and the rear end contour detection frame of the carriage based on the loaded cargo detection frame and the carriage rear end contour detection frame obtained by the recognition module. The loading rate calculation module is used to obtain the actual cargo length K of each truck by using the actual distance L between the loaded cargo detection frame and the cargo box rear contour detection frame of the distance conversion module, and to calculate the truck loading rate as (KL) / K * 100%. The acquisition module, annotation module, training module, recognition module, distance conversion module, and load rate calculation module are connected in sequence.

5. A system for calculating load rate based on image recognition according to a method according to claim 4, characterized in that: The acquisition module is an IP camera installed at the top of the rear of the truck bed or a camera fixed to the steering wheel to capture video of the truck loading scene.

6. A system for calculating load rate based on image recognition according to a method according to claim 4, characterized in that: The actual distance L between the loaded cargo detection frame and the rear contour detection frame of the carriage obtained by the distance conversion module is transmitted to the loading rate calculation module in JSON format.

7. A system for calculating load rate based on image recognition according to a method according to claim 4, characterized in that: The acquisition module also includes several infrared ranging sensors; the infrared ranging sensors are deployed at the front and rear edges of the truck bed along the longitudinal direction.

8. A system for calculating load rate based on image recognition according to a method according to claim 4, characterized in that: The annotation file is an XML file.

9. A system for calculating load rate based on image recognition according to a method according to claim 4, characterized in that: The YOLOv3 model includes the Darknet-53 network.

Images