A method for measuring the volume of a stack in a regular cubic space based on computer vision
By using computer vision technology for image processing and deep learning within a regular cubic space, the problems of accuracy and cost in stacking volume measurement have been solved, enabling fast and reliable volume calculation and improving production efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA WEST CONSTR ACAD OF BUILDING MATERIALS CO LTD
- Filing Date
- 2023-03-31
- Publication Date
- 2026-07-03
AI Technical Summary
Existing technologies for measuring stack volume suffer from problems such as large human error, high equipment costs, and susceptibility to environmental interference, especially in large cubic spaces where accurate and efficient calculations are difficult to achieve.
A computer vision-based approach is adopted to acquire projected images within a regular cube space, perform distortion correction, construct a deep learning neural network for semantic segmentation, and calculate the stacking volume using correction coefficients, thus avoiding the shortcomings of manual observation and high-cost instrument measurement.
It enables accurate, efficient, automated, and rapid calculation of stacking volume within a regular cubic space, improving industrial production efficiency and avoiding human error and environmental interference.
Smart Images
Figure CN116399228B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision, and in particular to a method for calculating the stacking volume within a regular cubic space based on computer vision. Background Technology
[0002] Currently, many industries use stacked storage of materials, which can make full use of warehouse space and improve storage efficiency and capacity. At present, the main methods for calculating the volume of stacked materials are: (1) Manual observation method: using human eyes to observe the height and shape of the stack and estimating the volume based on experience or formulas. This method is simple and easy to implement, but it is not accurate and scientific enough, and is easily affected by human error and subjective judgment. (2) Instrument measurement method: using instruments, such as ultrasound, laser, infrared, etc., to measure and analyze the height and shape of the stack to calculate the volume of the stack. This method is relatively accurate and scientific, but it requires high equipment costs and maintenance costs, and is easily affected by environmental factors such as humidity, temperature, and dust. Therefore, the market urgently needs a fast and low-cost method for calculating the volume of stacked materials.
[0003] Computer vision, an important branch of artificial intelligence, combines knowledge from multiple fields such as image processing, pattern recognition, and machine learning, and has broad application prospects. In industrial applications, computer vision has achieved many mature applications, such as item recognition, defect detection, and security inspection on automated production lines in factories.
[0004] CN201910972066.X discloses a method for measuring the volume of stacked materials based on a binocular camera, which uses a semi-global matching algorithm to obtain a standard point cloud for volume calculation. However, this method is not applicable to large cubic spaces with a depth of tens of meters. Summary of the Invention
[0005] This disclosure provides a novel algorithm for calculating stacking volume, which can accurately and efficiently calculate the stacking volume in a regular cubic space in real time.
[0006] The method for calculating the stacking volume within a regular cubic space disclosed herein includes the following steps:
[0007] S1, Obtain the complete projection image of the stack on both sides of the regular cube;
[0008] S2, perform distortion correction processing on the acquired images of the two side walls;
[0009] S3. Construct a deep learning neural network and use the processed image to perform semantic segmentation pre-training for stacked image semantic segmentation in a regular cube space to obtain a semantic segmentation model.
[0010] S4. The processed image is imported into the semantic segmentation model for image analysis to obtain the projected areas A1 and A2 of the stacked objects on the two side walls.
[0011] S5. Calculate the stacking volume V based on the projected areas A1 and A2, and the correction factor β. C :
[0012]
[0013] Where w is the width of the regular cube space.
[0014] Further, step S2 specifically includes:
[0015] Obtain the distortion coefficient of the camera lens;
[0016] Transform the image's pixel coordinate system to the camera coordinate system using an intrinsic parameter matrix;
[0017] Based on the camera distortion coefficient, distortion correction is performed in the camera coordinate system by pre-distortion, post-distortion, or simultaneous pre-distortion and post-distortion.
[0018] After the distortion correction operation is completed, the camera coordinate system is transformed back to the image pixel coordinate system.
[0019] Furthermore, step S3 specifically includes:
[0020] Images are acquired and preprocessed, and the stacking is segmented and labeled in each image using a labeling tool. Finally, the images are divided into training set, validation set and test set.
[0021] Constructing a deep learning neural network includes: using convolutional layers to extract high-level feature information from a stacked image, while using max pooling layers for downsampling to reduce the size of the feature map, and recording the pooling index of each pooling layer, wherein the pooling index is the position of the maximum value in each pooling region;
[0022] The stacked feature map is received from the encoder network composed of the above convolutional layers. At the same time, pooling index is used for upsampling to restore the size of the feature map while preserving spatial information. Finally, a feature map with the same size as the input image is obtained.
[0023] Convolutional layers are used to classify each pixel in the stacked image, that is, to assign it to a specific category;
[0024] The performance of the stacked image semantic segmentation model was evaluated using a validation set, and the above steps were repeated by adjusting the hyperparameters until a stacked image semantic segmentation model that meets the requirements was obtained.
[0025] Evaluate the robustness of the model using a test set, and repeat the above steps until the robustness of the stacked image semantic segmentation model meets the requirements.
[0026] Furthermore, in step S3, the loss function is one or more of the following: cross-entropy loss function, weighted cross-entropy loss function, DICE coefficient loss function, IOU loss function, and Tversky loss function.
[0027] Furthermore, the correction factor β is determined by the stacking volume V obtained from each measurement. C With respect to the actual volume V of the stack T The error calculation yielded the following:
[0028]
[0029] Where n represents the number of trials.
[0030] Further, step S4 specifically includes:
[0031] Import the stacking image data and user-defined parameters into the model data that has already been pre-trained for stacking semantic segmentation;
[0032] The semantic segmentation model is used to segment the projected images of both sides of the stack.
[0033] The calculation results are then optimized using a loss function.
[0034] The proportions of the stack in the two images are a1 and a2, respectively.
[0035] Calculate the projected area of the output processing results:
[0036] A i = i ×l×h
[0037] Where i = 1, 2; l and h are the length and height of the regular cube space, respectively.
[0038] Furthermore, in step S1, one or more of the following are used: a rotatable camera installed on the top of the regular cube space, a fixed camera symmetrically installed on both sides of the regular cube space, or a rotating camera, to obtain complete projected images of the stacked objects on both sides of the regular cube.
[0039] Compared with the prior art, the beneficial effects of this disclosure are: (1) It avoids the problem that the traditional manual observation method, which uses human eyes to observe the height and shape of the stack, is easily affected by human error and subjective judgment; (2) It avoids the disadvantages of the instrument measurement method, which uses instruments to measure the height and shape of the stack, which has high equipment costs and maintenance costs and is easily affected by environmental factors; (3) It can accurately and efficiently calculate the stack volume in a regular cubic space in real time; (4) It is automated, fast and reliable, and can effectively improve industrial production efficiency. Attached Figure Description
[0040] The above and other objects, features and advantages of this disclosure will become more apparent from the more detailed description of exemplary embodiments of this disclosure taken in conjunction with the accompanying drawings, in which the same reference numerals generally represent the same components.
[0041] Figure 1 A flowchart according to an exemplary embodiment is shown. Detailed Implementation
[0042] Preferred embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art.
[0043] This disclosure provides a method for calculating the stacking volume within a regular cubic space. A flowchart illustrating an exemplary embodiment of this disclosure is attached. Figure 1 As shown, the main steps include:
[0044] S101. Prepare a stacking volume measurement device, which consists of a camera, a computer 1 that receives camera image data, and a computer 2 that analyzes the image data.
[0045] The camera can be a rotatable camera mounted on the top of a regular cube space, a fixed camera symmetrically mounted on both sides of a regular cube space, or a rotating camera.
[0046] S102. Let the spatial length of the measured regular cube be l, the width be w, the height be h, and the actual stacking volume be V. T .
[0047] S103. Acquire complete projection images of the stacked objects on both sides of the regular cube using a camera and store them in computer 1.
[0048] S104. Transfer the stored images in computer 1 to computer 2 in chronological order, and use computer 2 to perform distortion correction processing on the acquired images of the two side walls.
[0049] Specifically, the following steps are included:
[0050] S4-1. Obtain the distortion coefficient of the camera lens;
[0051] S4-2. Transform the image's pixel coordinate system to the camera coordinate system using an intrinsic parameter matrix;
[0052] S4-3. Based on the camera distortion coefficient, perform distortion correction operations in the camera coordinate system by using pre-distortion, post-distortion, or both pre-distortion and post-distortion simultaneously.
[0053] S4-4 After the distortion correction operation is completed, the camera coordinate system is transformed back to the image pixel coordinate system.
[0054] S105. Construct a deep learning neural network, use the processed image to perform semantic segmentation pre-training of stacked images in a regular cube space, and deploy the resulting semantic segmentation model on computer 2.
[0055] Specific methods include:
[0056] S5-1. Acquire and enhance the images, and use the annotation tool to segment and label each image. Finally, divide them into training set, validation set and test set.
[0057] S5-2. Use convolutional layers to extract high-level feature information from the stacked image, and use max pooling layers to downsample and reduce the size of the feature map. At the same time, record the pooling index of each pooling layer (the position of the maximum value in each pooling region).
[0058] S5-3: Receive stacked feature maps from the encoder network, and simultaneously upsample using pooling indexes to restore the size of the feature maps while preserving spatial information, finally obtaining a feature map of the same size as the input image; the encoder network is a general term for a specific structure composed of the above convolutional layers, which preserves important information in the input image data while compressing dimensions, and is part of the neural network program;
[0059] S5-4. Use convolutional layers to classify each pixel in the stacked image, that is, assign it to a specific category.
[0060] S5-5. Use the validation set to evaluate the performance of the stacked image semantic segmentation model, and repeat steps S5-2 to S5-4 by adjusting hyperparameters (such as learning rate, weight decay, etc.) until a stacked image semantic segmentation model that meets the requirements is obtained.
[0061] S5-6. Use the test set to evaluate the robustness of the model. Repeat steps S5-2 to S5-5 until the robustness of the stacked image semantic segmentation model meets the requirements.
[0062] During the training process described above, the loss function can be cross-entropy loss function, weighted cross-entropy loss function, DICE coefficient loss function, IOU loss function, and Tversky loss function, etc.
[0063] S106. Import the processed image into the semantic segmentation model in computer 2 for image analysis to obtain the projected areas A1 and A2 of the stack on the two side walls.
[0064] Specifically, the following steps are included:
[0065] S6-1. Import the stacking image data and user-defined parameters into the model data that has been pre-trained for stacking semantic segmentation in computer 2.
[0066] S6-2. Use a semantic segmentation model to segment the projected images of both sides of the stack acquired by the camera;
[0067] S6-3. Perform loss function optimization on the calculation results;
[0068] S6-4. The proportions of the stacking in the two images are a1 and a2, respectively.
[0069] S6-4. Calculate the projected area of the output processing results:
[0070] A i = i ×l×h
[0071] Where i = 1, 2.
[0072] S107. Based on the projected areas A1 and A2, and the correction factor β, calculate the stacking volume V in computer 2. C :
[0073]
[0074] Preferably, the correction factor β is calculated from the error between the volume VC obtained from each measurement and the true volume VT:
[0075]
[0076] Where n represents the number of trials.
[0077] The above technical solutions are merely exemplary embodiments of the present invention. For those skilled in the art, based on the application methods and principles disclosed in the present invention, it is easy to make various types of improvements or modifications, and not limited to the methods described in the specific embodiments of the present invention. Therefore, the methods described above are merely preferred and not restrictive.
Claims
1. A method for calculating the stacking volume within a regular cubic space, comprising the following steps: S1, Obtain the complete projection image of the stack on both sides of the regular cube; S2, perform distortion correction processing on the acquired images of the two side walls; S3. Construct a deep learning neural network and use the processed image to perform semantic segmentation pre-training for stacked image semantic segmentation in a regular cube space to obtain a semantic segmentation model. S4. The processed image is imported into the semantic segmentation model for image analysis to obtain the projected areas A1 and A2 of the stacked objects on the two side walls. S5. Calculate the stacking volume V based on the projected areas A1 and A2, and the correction factor β. C : Where w is the width of the regular cube space.
2. The method according to claim 1, characterized in that, Step S2 specifically includes: Obtain the distortion coefficient of the camera lens; Transform the image's pixel coordinate system to the camera coordinate system using an intrinsic parameter matrix; Based on the camera distortion coefficient, distortion correction is performed in the camera coordinate system by pre-distortion, post-distortion, or simultaneous pre-distortion and post-distortion. After the distortion correction operation is completed, the camera coordinate system is transformed back to the image pixel coordinate system.
3. The method according to claim 1, characterized in that, Step S3 specifically includes: Images are acquired and preprocessed, and the stacking is segmented and labeled in each image using a labeling tool. Finally, the images are divided into training set, validation set and test set. Constructing a deep learning neural network includes: using convolutional layers to extract high-level feature information from a stacked image, while using max pooling layers for downsampling to reduce the size of the feature map, and recording the pooling index of each pooling layer, wherein the pooling index is the position of the maximum value in each pooling region; The stacked feature map is received from the encoder network composed of the above convolutional layers. At the same time, pooling index is used for upsampling to restore the size of the feature map while preserving spatial information. Finally, a feature map with the same size as the input image is obtained. Convolutional layers are used to classify each pixel in the stacked image, that is, to assign it to a specific category; The performance of the stacked image semantic segmentation model was evaluated using a validation set, and the above steps were repeated by adjusting the hyperparameters until a stacked image semantic segmentation model that meets the requirements was obtained. Evaluate the robustness of the model using a test set, and repeat the above steps until the robustness of the stacked image semantic segmentation model meets the requirements.
4. The method as described in claim 3, characterized in that, In step S3, the loss function is one or more of the following: cross-entropy loss function, weighted cross-entropy loss function, DICE coefficient loss function, IOU loss function, and Tversky loss function.
5. The method according to claim 1, characterized in that, The correction factor β is determined by the stack volume V obtained from each measurement. C With respect to the actual volume V of the stack T The error calculation yielded the following: Where n represents the number of trials.
6. The method according to claim 1, characterized in that, Step S4 specifically includes: Import the stacking image data and user-defined parameters into the model data that has already been pre-trained for stacking semantic segmentation; The semantic segmentation model is used to segment the projected images of both sides of the stack. The calculation results are then optimized using a loss function. The proportions of the stack in the two images are a1 and a2, respectively. Calculate the projected area of the output processing results: A i =a i ×l×h Where i = 1, 2; l and h are the length and height of the regular cube space, respectively.
7. The method according to any one of claims 1-6, characterized in that, In step S1, one or more of the following are used: a rotatable camera installed on the top of the regular cube space, a fixed camera symmetrically installed on both sides of the regular cube space, or a rotating camera, to obtain complete projected images of the stacked objects on both sides of the regular cube.