Cargo volume estimation method, device, electronic device and storage medium

Abstract

The present invention provides a cargo volume estimation method, device, electronic device, and storage medium, relating to the field of logistics technology. The method includes: obtaining a structured light 3D point cloud, a radar point cloud, and an image of the cargo to be estimated; performing registration and fusion processing on the structured light 3D point cloud, the radar point cloud, and the image to obtain fused data; determining the target type of the cargo to be estimated based on the fused data; and inputting the fused data into a target volume estimation model corresponding to the target type to obtain the volume of the cargo to be estimated. After obtaining the fused data of the cargo to be estimated, the present invention first determines the target type of the cargo to be estimated based on the fused data, and then inputs the fused data into a target volume estimation model corresponding to the target type to perform volume estimation. The target volume estimation model corresponding to the target type is specifically trained based on sample fusion data of a first sample cargo corresponding to the target type, thereby improving the accuracy of cargo volume estimation.

Description

Technical Field

[0001] The present invention relates to the field of logistics technology, and in particular to a cargo volume estimation method, device, electronic equipment and storage medium. Background Art

[0002] In the logistics sector, accurate cargo volume measurement is crucial for optimizing transportation planning, ensuring reasonable billing, and efficient warehouse management. With the rapid development of e-commerce and the increasing complexity of supply chains, accurate volume data has become a key component in improving logistics efficiency and reducing costs.

[0003] In related technologies, cargo images are typically captured and fed into a volume estimation model. The model then extracts features, such as edges, from the cargo images to estimate the cargo volume. However, these volume estimation models are difficult to adapt to different cargo types, reducing the accuracy of cargo volume estimation. Summary of the Invention

[0004] The present invention provides a cargo volume estimation method, device, electronic device and storage medium, which are used to solve the defect of reducing the accuracy of cargo volume estimation in the prior art.

[0005] The present invention provides a cargo volume estimation method, comprising the following steps.

[0006] Acquire multimodal data of the goods to be estimated, the multimodal data including a structured light 3D point cloud of the goods to be estimated, a radar point cloud of the goods to be estimated, and an image of the goods to be estimated;

[0007] performing registration and fusion processing on the structured light 3D point cloud, the radar point cloud, and the image to obtain fused data;

[0008] determining the target type of the cargo to be estimated based on the fused data;

[0009] The fused data is input into a target volume estimation model corresponding to the target type to obtain the volume of the cargo to be estimated output by the target volume estimation model, where the target volume estimation model is trained based on the sample fusion data of a first sample cargo corresponding to the target type.

[0010] According to a cargo volume estimation method provided by the present invention, the target volume estimation model is trained based on the following method:

[0011] Determining first model structure parameters and first model training parameters corresponding to the target type based on the correspondence between the type, model structure parameters, and model training parameters;

[0012] Determining a first initial volume estimation model based on the first model structure parameters and the first model training parameters;

[0013] Inputting the sample fusion data of the first sample cargo into the first initial volume estimation model to obtain a first predicted volume of the first sample cargo output by the first initial volume estimation model;

[0014] Based on the first predicted volume and the volume label corresponding to the first sample cargo, the first model structure parameters and the first model training parameters are iteratively adjusted to obtain the target volume estimation model.

[0015] According to a cargo volume estimation method provided by the present invention, inputting the fused data into a target volume estimation model corresponding to the target type to obtain the volume of the cargo to be estimated output by the target volume estimation model includes:

[0016] Determining a target logistics scenario for the goods to be estimated;

[0017] The fused data is input into a target volume estimation model corresponding to the target type and the target logistics scenario to obtain the volume of the cargo to be estimated output by the target volume estimation model.

[0018] According to a cargo volume estimation method provided by the present invention, the target volume estimation model is trained based on the following method:

[0019] Determine the second model structure parameters and the second model training parameters corresponding to the target type and the target logistics scenario based on the correspondence between the logistics scenario, type, model structure parameters and model training parameters;

[0020] determining a second initial volume estimation model based on the second model structure parameters and the second model training parameters;

[0021] Inputting the sample fusion data of the first sample cargo into the second initial volume estimation model to obtain a second predicted volume of the first sample cargo output by the second initial volume estimation model;

[0022] Based on the second predicted volume and the volume label corresponding to the first sample cargo, the second model structure parameters and the second model training parameters are iteratively adjusted to obtain the target volume estimation model.

[0023] According to a cargo volume estimation method provided by the present invention, the registering and fusing the structured light 3D point cloud, the radar point cloud, and the image to obtain fused data includes:

[0024] Performing denoising on the structured light 3D point cloud and the radar point cloud based on a statistical filtering algorithm to obtain a denoised structured light 3D point cloud and a denoised radar point cloud;

[0025] Performing denoising on the image based on a Gaussian filtering algorithm to obtain a denoised image;

[0026] The denoised structured light 3D point cloud, the denoised radar point cloud and the denoised image are registered and fused to obtain the fused data.

[0027] According to a cargo volume estimation method provided by the present invention, determining the target type of the cargo to be estimated based on the fused data includes:

[0028] The fused data is input into a clustering model to obtain the target type of the goods to be estimated output by the clustering model, wherein the clustering model is trained in an unsupervised manner based on the sample fused data of each of the different types of second sample goods.

[0029] According to a cargo volume estimation method provided by the present invention, the target volume estimation model includes a convolution module, a pooling layer and a fully connected layer connected in sequence, and the convolution module includes multiple 3D convolution layers connected in sequence.

[0030] The present invention also provides a cargo volume estimation device, comprising:

[0031] an acquisition unit, configured to acquire multimodal data of the goods to be estimated, the multimodal data including a structured light 3D point cloud of the goods to be estimated, a radar point cloud of the goods to be estimated, and an image of the goods to be estimated;

[0032] a registration and fusion unit, configured to perform registration and fusion processing on the structured light 3D point cloud, the radar point cloud, and the image to obtain fused data;

[0033] a determining unit, configured to determine a target type of the cargo to be estimated based on the fused data;

[0034] an estimation unit, configured to input the fused data into a target volume estimation model corresponding to the target type, to obtain the volume of the cargo to be estimated output by the target volume estimation model, wherein the target volume estimation model is trained based on the sample fused data of a first sample cargo corresponding to the target type.

[0035] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any of the above-described cargo volume estimation methods is implemented.

[0036] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described cargo volume estimation methods.

[0037] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned cargo volume estimation methods.

[0038] The cargo volume estimation method, device, electronic device, and storage medium provided by the present invention obtain a structured light 3D point cloud of the cargo to be estimated, a radar point cloud of the cargo to be estimated, and an image of the cargo to be estimated, perform registration and fusion processing on the structured light 3D point cloud, radar point cloud, and image to obtain fused data, determine the target type of the cargo to be estimated based on the fused data, input the fused data into a target volume estimation model trained based on sample fusion data of a first sample cargo corresponding to the target type, and obtain the volume of the cargo to be estimated output by the target volume estimation model. It can be seen that after obtaining the fused data of the cargo to be estimated, the present invention first determines the target type of the cargo to be estimated based on the fused data, and then inputs the fused data into the target volume estimation model corresponding to the target type for volume estimation. The target volume estimation model corresponding to the target type is specifically trained based on the sample fusion data of the first sample cargo corresponding to the target type, thereby improving the accuracy of cargo volume estimation. In addition, the multimodal data collected by the present invention includes the structured light 3D point cloud, radar point cloud, and image of the cargo to be estimated, which improves the integrity of cargo information collection and further improves the accuracy of cargo volume estimation. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0040] Figure 1 It is a flowchart of a cargo volume estimation method provided by an embodiment of the present invention.

[0041] Figure 2 This is one of the flowcharts of the target volume estimation model training method provided by an embodiment of the present invention.

[0042] Figure 3 This is the second flowchart of the target volume estimation model training method provided by an embodiment of the present invention.

[0043] Figure 4 This is an overall framework diagram of cargo volume estimation provided by an embodiment of the present invention.

[0044] Figure 5 2 is a schematic diagram of the structure of a cargo volume estimation device provided by an embodiment of the present invention.

[0045] Figure 6 It is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0047] The following combination Figures 1-4 The cargo volume estimation method of the present invention is described. The method can be implemented by an electronic device such as a terminal, tablet computer, computer, or server, or by a cargo volume estimation device within the electronic device. The cargo volume estimation device can be implemented using software, hardware, or a combination of both.

[0048] Figure 1 FIG. 1 is a flow chart of a cargo volume estimation method provided by an embodiment of the present invention. Figure 1 As shown, the cargo volume estimation method includes the following steps:

[0049] Step 101: Acquire multimodal data of goods to be estimated, where the multimodal data includes a structured light 3D point cloud of the goods to be estimated, a radar point cloud of the goods to be estimated, and an image of the goods to be estimated.

[0050] For example, a multimodal data acquisition module integrating structured light 3D cameras, lidar, and RGB cameras is constructed. These cameras are deployed at key logistics nodes, such as warehouse entrances and exits, cargo sorting areas, and inside transport vehicles. The structured light 3D cameras project specific patterns (e.g., stripes or grids) onto the cargo to be estimated and analyze the deformation of the reflected light to obtain depth information about the cargo surface. This technology provides high-resolution surface details of objects at close range. The data collected by the structured light 3D cameras is called a structured light 3D point cloud. The lidar uses laser pulses to measure the distance to the cargo to be estimated, quickly acquiring the approximate outline of the cargo within a large range. This is suitable for long-distance and large-area scenarios. The data collected by the lidar is called a radar point cloud. The RGB cameras capture images of the cargo to be estimated, including color and texture information. These images help identify the cargo's boundaries and features. Structured light 3D cameras, lidar, and RGB cameras work together to comprehensively acquire multimodal data of the cargo to be estimated at different scales and angles, effectively making up for the shortcomings of a single sensor.

[0051] Step 102: performing registration and fusion processing on the structured light 3D point cloud, the radar point cloud, and the image to obtain fused data.

[0052] For example, after obtaining a structured light 3D point cloud, a radar point cloud, and an image of the goods to be estimated, a feature-based registration algorithm is used to extract feature points from the structured light 3D point cloud and the radar point cloud. For example, the extracted feature points include edge points and corner points. Registration methods such as iterative closest point (ICP) are then used to match and align the extracted feature points, thereby fusing the multi-source data to generate a more complete and accurate fused point cloud data of the goods to be estimated. This fused point cloud data is then registered with an image captured by an RGB camera, using texture and color features in the image to assist in determining the boundaries and shape of the goods to be estimated, resulting in the final fused data. This fused data more accurately represents the goods to be estimated.

[0053] Step 103: Determine the target type of the cargo to be estimated based on the fused data.

[0054] For example, after obtaining the fused data of the goods to be estimated, the similarity between the fused data and each reference fused data stored in a goods type database can be calculated. The goods type corresponding to the reference fused data with the highest similarity is determined as the target type of the goods to be estimated. The goods type database stores the correspondence between the reference fused data and the goods type. The reference fused data is determined based on the reference structured light 3D point cloud, reference radar point cloud, and reference image of the reference goods. Furthermore, the goods types mentioned here can be categorized as small, medium, and large based on their size. For example, letters and documents are small, clothes and shoes are medium, and televisions and washing machines are large. Goods types can also be categorized as regular and irregular based on their shape. For example, goods packed in cartons are regular, while goods packed in sacks are irregular. This is not a limitation of the present invention.

[0055] Step 104: Input the fused data into a target volume estimation model corresponding to the target type to obtain the volume of the cargo to be estimated output by the target volume estimation model, where the target volume estimation model is trained based on the sample fused data of the first sample cargo corresponding to the target type.

[0056] For example, after determining the target type of the goods to be estimated, based on the correspondence between the type and the volume estimation model, the target volume estimation model corresponding to the target type is called, and the fused data of the goods to be estimated is input into the target volume estimation model corresponding to the target type. The input fused data is subjected to feature analysis by the target volume estimation model to estimate the volume of the goods to be estimated.

[0057] The cargo volume estimation method provided by the present invention obtains a structured light 3D point cloud of the cargo to be estimated, a radar point cloud of the cargo to be estimated, and an image of the cargo to be estimated, performs registration and fusion processing on the structured light 3D point cloud, the radar point cloud, and the image to obtain fused data, determines the target type of the cargo to be estimated based on the fused data, inputs the fused data into a target volume estimation model trained based on sample fusion data of a first sample cargo corresponding to the target type, and obtains the volume of the cargo to be estimated output by the target volume estimation model. It can be seen that after obtaining the fused data of the cargo to be estimated, the present invention first determines the target type of the cargo to be estimated based on the fused data, and then inputs the fused data into the target volume estimation model corresponding to the target type for volume estimation. The target volume estimation model corresponding to the target type is specifically trained based on the sample fusion data of the first sample cargo corresponding to the target type, thereby improving the accuracy of cargo volume estimation. In addition, the multimodal data collected by the present invention includes the structured light 3D point cloud, the radar point cloud, and the image of the cargo to be estimated, which improves the integrity of cargo information collection, can effectively deal with all cargo types, background complexity, and occlusion problems in logistics scenarios, and further improves the accuracy of cargo volume estimation.

[0058] In one embodiment, Figure 2 FIG. 1 is a flow chart of a training method for a target volume estimation model provided by an embodiment of the present invention. Figure 2 As shown, the target volume estimation model is trained based on the following method:

[0059] Step 201: Determine first model structure parameters and first model training parameters corresponding to the target type based on the correspondence between the type, model structure parameters and model training parameters.

[0060] For example, model structure parameters and model training parameters are predefined for each type of cargo. Model structure parameters may include convolution kernel size, number of convolution layers, and network depth. Model training parameters may include learning rate and regularization coefficient. Taking cargo type classification based on cargo size as an example, the convolution kernel size for small cargo can be predefined as 3x3, the number of convolution layers as 3, the network depth as shallow, the learning rate as 0.001, and the regularization coefficient as 0.01; the convolution kernel size for medium cargo can be predefined as 5x5, the number of convolution layers as 5, the network depth as medium, the learning rate as 0.0005, and the regularization coefficient as 0.005; and the convolution kernel size for large cargo can be predefined as 7x7, the number of convolution layers as 7, the network depth as deep, the learning rate as 0.0001, and the regularization coefficient as 0.001. Based on the correspondence between type, model structure parameters, and model training parameters, the first model structure parameters and first model training parameters corresponding to the target type can be found. For example, if the target type is small cargo, the first model structure parameters include a convolution kernel size of 3x3, 3 convolution layers, a shallow network depth, and the first model training parameters include a learning rate of 0.001 and a regularization coefficient of 0.01.

[0061] Step 202: Determine a first initial volume estimation model based on the first model structure parameters and the first model training parameters.

[0062] For example, when the first model structure parameters and the first model training parameters are obtained, the model architecture can be defined according to the first model structure parameters, and the model training process can be configured according to the first model training parameters, and finally, a first initial volume estimation model can be constructed.

[0063] Step 203: Input the sample fusion data of the first sample cargo into the first initial volume estimation model to obtain a first predicted volume of the first sample cargo output by the first initial volume estimation model.

[0064] For example, for each first sample of a target type, a structured light 3D camera projects a specific pattern (e.g., stripes or a grid) onto the first sample and analyzes the deformation of the reflected light to obtain depth information about the surface of the first sample. The data collected by the structured light 3D camera is referred to as a sample structured light 3D point cloud. A lidar (LiDAR) uses laser pulses to measure the distance to the first sample, quickly acquiring the approximate outline of the first sample within a large area. The data collected by the LiDAR is referred to as a sample radar point cloud. An RGB camera captures a sample image of the first sample. The sample image includes color and texture information, which assists in identifying the boundaries and features of the first sample. The sample structured light 3D point cloud, the sample radar point cloud, and the sample image are fused and registered to generate sample fused data for the first sample. This sample fused data is then input into the constructed first initial volume estimation model, which performs feature analysis on the input sample fused data to estimate a first predicted volume for the first sample.

[0065] Step 204: Based on the first predicted volume and the volume label corresponding to the first sample cargo, iteratively adjust the first model structure parameters and the first model training parameters to obtain the target volume estimation model.

[0066] For example, when the first predicted volume of the first sample cargo output by the first initial volume estimation model is obtained, the first loss function is determined based on the following formula (1): , the first loss function is the mean square error loss function, based on the first loss function The first model structure parameters and the first model training parameters are iteratively adjusted until a convergence condition is reached, and finally a target volume estimation model is obtained, so that the prediction result of the target volume estimation model is closer to the actual volume.

[0067]

[0068] in, Indicates the The first predicted volume of the first sample cargo, Indicates the Volume labels for the first sample shipment, Indicates the quantity of the first sample goods.

[0069] In this embodiment, the correspondence between types, model structure parameters and model training parameters is pre-stored. For each type of goods, the model structure parameters and model training parameters corresponding to the type are determined based on the correspondence, and an initial volume estimation model of the type is constructed based on the model structure parameters and model training parameters corresponding to the type. The initial volume estimation model is then trained based on sample fusion data of sample goods of the type, and finally a target volume estimation model of the type is obtained, so that the target volume estimation model of the type can be fully adapted to the goods of the type. The target volume estimation model corresponding to each type can be obtained by using the same method. Through this adaptive model optimization mechanism, the combination of target volume estimation models corresponding to each type can dynamically adapt to different types of goods, thereby improving the generalization ability and adaptability of the model.

[0070] In one embodiment, step 104 inputs the fused data into a target volume estimation model corresponding to the target type to obtain the volume of the cargo to be estimated output by the target volume estimation model. This can be specifically achieved by:

[0071] Determine the target logistics scenario in which the cargo to be estimated is located; input the fused data into a target volume estimation model corresponding to the target type and the target logistics scenario, and obtain the volume of the cargo to be estimated output by the target volume estimation model.

[0072] For example, the environment in which the goods to be estimated are located is photographed by a camera to obtain an environmental image. Based on the analysis of the environmental image, the target logistics scene in which the goods to be estimated are located is determined. The logistics scenes can be classified based on the means of transportation. For example, logistics scenes include air transportation scenes, road transportation scenes, and sea transportation scenes. For example, if the goods to be estimated are located on an airplane, the target logistics scene in which the goods to be estimated are located is an air transportation scene. Based on the correspondence between the type of logistics scene and the volume estimation model, the target volume estimation model corresponding to the target type and target logistics scene is called, and the fused data of the goods to be estimated is input into the target volume estimation model corresponding to the target type and target logistics scene. The target volume estimation model performs feature analysis on the input fused data to estimate the volume of the goods to be estimated.

[0073] In this embodiment, the target volume estimation model is determined based on the target type of the goods to be estimated and the target logistics scenario, so that the target volume estimation model can fully adapt to the needs of specific goods types and specific logistics scenarios, further improving the accuracy of cargo volume estimation.

[0074] In one embodiment, Figure 3 This is the second flow chart of the training method of the target volume estimation model provided by the embodiment of the present invention. Figure 3As shown, the target volume estimation model is trained based on the following method:

[0075] Step 301: Based on the correspondence between the logistics scenario, type, model structure parameters and model training parameters, determine the second model structure parameters and second model training parameters corresponding to the target type and the target logistics scenario.

[0076] For example, model structure parameters and model training parameters are predefined for each logistics scenario and cargo type. Model structure parameters may include convolution kernel size, number of convolution layers, and network depth, while model training parameters may include learning rate and regularization coefficient. Taking the logistics scenario based on transportation vehicle classification as an example, the convolution kernel size for air transport scenarios can be predefined as 3x3, the number of convolution layers as 4, the network depth as medium, the learning rate as 0.0008, and the regularization coefficient as 0.008; the convolution kernel size for road transport scenarios can be predefined as 5x5, the number of convolution layers as 5, the network depth as medium, the learning rate as 0.0005, and the regularization coefficient as 0.005; and the convolution kernel size for maritime transport scenarios can be predefined as 7x7, the number of convolution layers as 6, the network depth as deep, the learning rate as 0.0002, and the regularization coefficient as 0.002. Based on the correspondence between type, logistics scenario, model structure parameters, and model training parameters, the second model structure parameters and second model training parameters corresponding to the target type and target logistics scenario can be found. For example, if the target type is medium-sized cargo and the target logistics scenario is road transport, the second model structure parameters include a convolution kernel size of 5x5, 5 convolution layers, and a medium network depth. The second model training parameters include a learning rate of 0.0005 and a regularization coefficient of 0.005. This scenario-based and type-specific parameter definition and matching method ensures that the target volume estimation model better adapts to the needs of specific cargo types and logistics scenarios, significantly improving the accuracy and practicality of volume estimation.

[0077] Step 302: Determine a second initial volume estimation model based on the second model structure parameters and the second model training parameters.

[0078] For example, when the second model structure parameters and the second model training parameters are obtained, the model architecture can be defined according to the second model structure parameters, and the model training process can be configured according to the second model training parameters, and finally, the second initial volume estimation model can be constructed.

[0079] Step 303: Input the sample fusion data of the first sample cargo into the second initial volume estimation model to obtain a second predicted volume of the first sample cargo output by the second initial volume estimation model.

[0080] For example, for each first sample of a target type, a structured light 3D camera projects a specific pattern (e.g., stripes or a grid) onto the first sample and analyzes the deformation of the reflected light to obtain depth information about the surface of the first sample. The data collected by the structured light 3D camera is referred to as a sample structured light 3D point cloud. A lidar (LiDAR) uses laser pulses to measure the distance to the first sample, quickly acquiring the approximate outline of the first sample within a large area. The data collected by the LiDAR is referred to as a sample radar point cloud. An RGB camera captures a sample image of the first sample. The sample image includes color and texture information, which assists in identifying the boundaries and features of the first sample. The sample structured light 3D point cloud, the sample radar point cloud, and the sample image are fused and registered to generate sample fused data for the first sample. This sample fused data is then input into the constructed second initial volume estimation model, which performs feature analysis on the input sample fused data to estimate the second predicted volume of the first sample.

[0081] Step 304: Based on the second predicted volume and the volume label corresponding to the first sample cargo, iteratively adjust the second model structure parameters and the second model training parameters to obtain the target volume estimation model.

[0082] For example, when the second predicted volume of the first sample cargo output by the second initial volume estimation model is obtained, the second loss function is determined based on a method similar to the above formula (1), and the second model structure parameters and the second model training parameters are iteratively adjusted based on the second loss function until the convergence condition is reached, and finally the target volume estimation model is obtained.

[0083] In this embodiment, the correspondence between logistics scenarios, types, model structure parameters and model training parameters is pre-stored. For each type of goods and logistics scenario, the model structure parameters and model training parameters corresponding to the type and logistics scenario are determined based on the correspondence, and an initial volume estimation model for the type and logistics scenario is constructed based on the model structure parameters and model training parameters corresponding to the type and logistics scenario. The initial volume estimation model is then trained based on sample fusion data of sample goods under the type and logistics scenario, and finally a target volume estimation model for the type and logistics scenario is obtained, so that the target volume estimation model for the type and logistics scenario can fully adapt to the goods of this type in the logistics scenario. Using the same method, the target volume estimation model corresponding to each type and logistics scenario can be obtained, so that the combination of target volume estimation models corresponding to each type and logistics scenario can adapt to different types of goods and different logistics scenarios, thereby improving the generalization ability and adaptability of the model.

[0084] In one embodiment, step 102 performs registration and fusion processing on the structured light 3D point cloud, the radar point cloud, and the image to obtain fused data, which can be specifically achieved by:

[0085] The structured light 3D point cloud and the radar point cloud are denoised based on a statistical filtering algorithm to obtain a denoised structured light 3D point cloud and a denoised radar point cloud; the image is denoised based on a Gaussian filtering algorithm to obtain a denoised image; and the denoised structured light 3D point cloud, the denoised radar point cloud, and the denoised image are registered and fused to obtain the fused data.

[0086] For example, the structured light 3D point cloud and the radar point cloud can be denoised based on the following formula (2), that is, a statistical filtering algorithm is used to remove outliers, and according to the distribution of points in the neighborhood of the point cloud, points that are significantly deviated from the surrounding points are judged and eliminated, and finally the denoised structured light 3D point cloud and the denoised radar point cloud are obtained.

[0087]

[0088] Among them, when denoising the structured light 3D point cloud, Represents the position of a point in the structured light 3D point cloud, Represents the denoised structured light 3D point cloud. When denoising the radar point cloud, represents the position of a point in the radar point cloud, represents the radar point cloud after denoising, express Neighborhood, express The mean of the midpoint positions, express The standard deviation of the positions of the interior points, Indicates the set threshold.

[0089] The image is denoised based on the following formula (3), that is, the Gaussian filter algorithm is used to remove noise to obtain the denoised image:

[0090]

[0091] in, represents the denoised image, represents the pixel coordinates in the image, Indicates the standard deviation, which is a parameter of the Gaussian filter. It is set according to the type and degree of noise to be filtered out. Controls the filtering strength.

[0092] In this embodiment, a statistical filtering algorithm can effectively remove outliers and noise points in the structured light 3D point cloud and radar point cloud, retaining the true geometric features of the cargo to be estimated. A Gaussian filtering algorithm can smooth the noise in the image of the cargo to be estimated, while retaining edge and texture information and improving image clarity. The denoised structured light 3D point cloud, denoised radar point cloud, and denoised image are then registered and fused, so that the final fused data can accurately represent the cargo to be estimated, further improving the accuracy of cargo volume estimation.

[0093] In one embodiment, the above step 103 determines the target type of the cargo to be estimated based on the fused data, which can be specifically implemented in the following manner:

[0094] The fused data is input into a clustering model to obtain the target type of the goods to be estimated output by the clustering model, wherein the clustering model is trained in an unsupervised manner based on the sample fused data of each of the different types of second sample goods.

[0095] For example, sample fusion data of the second sample cargo is obtained, including a denoised sample structured light 3D point cloud, a denoised sample radar point cloud, and a denoised image. The feature vectors of all the sample fusion data of the second sample cargo are input into an initial clustering model, which can be a K-Means model. The cluster centers are iteratively updated, and the feature vectors of the sample fusion data are assigned to the nearest cluster. The convergence of the initial clustering model is determined based on the characteristics of the algorithm. For example, the K-Means model stops iterations when the cluster centers no longer change, ultimately resulting in a clustering model trained using an unsupervised approach. When the fusion data of the cargo to be estimated is obtained, it is input into the trained clustering model, which is then used to analyze the fusion data of the cargo to be estimated, ultimately determining the target type of the cargo to be estimated.

[0096] In this embodiment, a clustering model is obtained by training in an unsupervised manner based on the sample fusion data of different types of second sample goods. The type of the goods to be estimated is identified through the clustering model, which reduces the dependence on large-scale labeled data, reduces the labeling cost, and also improves the accuracy of determining the type of goods.

[0097] In one embodiment, the target volume estimation model includes a convolution module, a pooling layer, and a fully connected layer connected in sequence, and the convolution module includes a plurality of 3D convolution layers connected in sequence.

[0098] For example, feature extraction and regression prediction are performed based on the fused data of the goods to be estimated through multiple 3D convolutional layers, pooling layers, and fully connected layers, and the volume of the goods to be estimated is finally obtained. The target volume estimation model of the present invention includes multiple 3D convolutional layers. For example, the convolution kernel size of the first 3D convolutional layer is 3 × 3 × 3, the step size is 1, and the ReLU activation function is used. The formula is f (x) = max (0, x) to perform preliminary feature extraction on the input fused data. As the network layer advances, the feature map gradually shrinks, and the extracted features become more abstract and advanced. Among them, x in the formula f (x) = max (0, x) represents the output value after the convolution operation on the fused data, and f (x) represents the value after processing by the ReLU activation function.

[0099] It should be noted that the present invention can also use a variant of the Recurrent Neural Network (RNN), such as a Long Short-Term Memory Network (LSTM) or a Gated Recurrent Unit (GRU) to process the fused data, instead of the 3D Convolutional Neural Network (3D-CNN) including a convolution module, a pooling layer and a fully connected layer. The present invention is not limited to this.

[0100] Figure 4 This is the overall framework diagram of cargo volume estimation provided by the embodiment of the present invention, such as Figure 4As shown, the system includes a multimodal data acquisition module, a data preprocessing and fusion module, a deep learning volume estimation module, and an adaptive model optimization module. The multimodal data acquisition module includes a structured light 3D camera, a lidar, and an RGB camera. The structured light 3D camera acquires a structured light 3D point cloud of the goods to be estimated, the lidar acquires a radar point cloud of the goods to be estimated, and the RGB camera acquires an image of the goods to be estimated. The data preprocessing and fusion module denoises the structured light 3D point cloud and radar point cloud using a statistical filtering algorithm to produce denoised structured light 3D point clouds and radar point clouds, denoises the image using a Gaussian filtering algorithm to produce a denoised image, and registers and fuses the denoised structured light 3D point cloud, radar point cloud, and image to produce fused data, which is used to represent the 3D model of the goods to be estimated. The deep learning volume estimation module is used to input the fused data into the volume estimation model, which then performs 3D-CNN estimation, feature extraction, and regression prediction. Ultimately, the volume estimation model outputs the estimated volume of the cargo to be estimated. Furthermore, an adaptive model optimization mechanism is introduced. Based on the correspondence between the logistics scenario, type, model structure parameters, and model training parameters, second model structure parameters and second model training parameters corresponding to the target type and target logistics scenario are determined. Based on these second model structure parameters and second model training parameters, a second initial volume estimation model is determined. The sample fused data of the first sample cargo is input into the second initial volume estimation model to obtain a second predicted volume of the first sample cargo output by the second initial volume estimation model. Based on the second predicted volume and the volume label corresponding to the first sample cargo, the second model structure parameters and second model training parameters are iteratively adjusted to obtain a target volume estimation model corresponding to the target type and target logistics scenario. Furthermore, the fused data is input into the target volume estimation model corresponding to the target type and target logistics scenario to estimate the volume of the cargo to be estimated, thereby improving the accuracy of cargo volume estimation.

[0101] The cargo volume estimation method provided by the embodiment of the present invention can more accurately identify the shape and boundaries of cargo through multimodal data fusion and deep learning models, effectively reduce volume estimation errors, and significantly improve the accuracy compared with traditional methods and single-sensor machine learning methods; and the multimodal data collected by the present invention can adapt to the complex environment, diversity of cargo types and occlusion conditions in logistics scenarios, and can be applied to volume measurement of different types of cargo and logistics links; in addition, a clustering model is obtained by unsupervised training based on the sample fusion data of different types of second sample cargo, and the type of cargo to be estimated is identified through the clustering model, which reduces the dependence on large-scale labeled data, reduces the labeling cost, and also improves the accuracy of cargo type determination.

[0102] The cargo volume estimation device provided by the present invention is described below. The cargo volume estimation device described below and the cargo volume estimation method described above can be referenced to each other.

[0103] Figure 5 FIG. 1 is a schematic diagram of the structure of a cargo volume estimation device provided by an embodiment of the present invention. Figure 5 As shown, the cargo volume estimation device 500 includes an acquisition unit 501, a registration and fusion unit 502, a determination unit 503 and an estimation unit 504; wherein:

[0104] An acquisition unit 501 is configured to acquire multimodal data of the goods to be estimated, wherein the multimodal data includes a structured light 3D point cloud of the goods to be estimated, a radar point cloud of the goods to be estimated, and an image of the goods to be estimated;

[0105] a registration and fusion unit 502 for performing registration and fusion processing on the structured light 3D point cloud, the radar point cloud, and the image to obtain fused data;

[0106] A determining unit 503 is configured to determine the target type of the cargo to be estimated based on the fused data;

[0107] An estimation unit 504 is configured to input the fused data into a target volume estimation model corresponding to the target type to obtain the volume of the cargo to be estimated output by the target volume estimation model, where the target volume estimation model is trained based on the sample fused data of the first sample cargo corresponding to the target type.

[0108] The cargo volume estimation device provided by the present invention obtains a structured light 3D point cloud of the cargo to be estimated, a radar point cloud of the cargo to be estimated, and an image of the cargo to be estimated, performs registration and fusion processing on the structured light 3D point cloud, the radar point cloud, and the image to obtain fused data, determines the target type of the cargo to be estimated based on the fused data, inputs the fused data into a target volume estimation model trained based on sample fusion data of a first sample cargo corresponding to the target type, and obtains the volume of the cargo to be estimated output by the target volume estimation model. It can be seen that after obtaining the fused data of the cargo to be estimated, the present invention first determines the target type of the cargo to be estimated based on the fused data, and then inputs the fused data into the target volume estimation model corresponding to the target type for volume estimation. The target volume estimation model corresponding to the target type is specifically trained based on the sample fusion data of the first sample cargo corresponding to the target type, thereby improving the accuracy of cargo volume estimation. In addition, the multimodal data collected by the present invention includes the structured light 3D point cloud, the radar point cloud, and the image of the cargo to be estimated, which improves the integrity of cargo information collection and further improves the accuracy of cargo volume estimation.

[0109] Based on any of the above embodiments, the target volume estimation model is trained based on the following method:

[0110] Determining first model structure parameters and first model training parameters corresponding to the target type based on the correspondence between the type, model structure parameters, and model training parameters;

[0111] Determining a first initial volume estimation model based on the first model structure parameters and the first model training parameters;

[0112] Inputting the sample fusion data of the first sample cargo into the first initial volume estimation model to obtain a first predicted volume of the first sample cargo output by the first initial volume estimation model;

[0113] Based on the first predicted volume and the volume label corresponding to the first sample cargo, the first model structure parameters and the first model training parameters are iteratively adjusted to obtain the target volume estimation model.

[0114] Based on any of the above embodiments, the estimating unit 504 is specifically configured to:

[0115] Determining a target logistics scenario for the goods to be estimated;

[0116] The fused data is input into a target volume estimation model corresponding to the target type and the target logistics scenario to obtain the volume of the cargo to be estimated output by the target volume estimation model.

[0117] Based on any of the above embodiments, the target volume estimation model is trained based on the following method:

[0118] Determine the second model structure parameters and the second model training parameters corresponding to the target type and the target logistics scenario based on the correspondence between the logistics scenario, type, model structure parameters and model training parameters;

[0119] determining a second initial volume estimation model based on the second model structure parameters and the second model training parameters;

[0120] Inputting the sample fusion data of the first sample cargo into the second initial volume estimation model to obtain a second predicted volume of the first sample cargo output by the second initial volume estimation model;

[0121] Based on the second predicted volume and the volume label corresponding to the first sample cargo, the second model structure parameters and the second model training parameters are iteratively adjusted to obtain the target volume estimation model.

[0122] Based on any of the above embodiments, the registration and fusion unit 502 is specifically configured to:

[0123] Performing denoising on the structured light 3D point cloud and the radar point cloud based on a statistical filtering algorithm to obtain a denoised structured light 3D point cloud and a denoised radar point cloud;

[0124] Performing denoising on the image based on a Gaussian filtering algorithm to obtain a denoised image;

[0125] The denoised structured light 3D point cloud, the denoised radar point cloud and the denoised image are registered and fused to obtain the fused data.

[0126] Based on any of the foregoing embodiments, the determining unit 503 is specifically configured to:

[0127] The fused data is input into a clustering model to obtain the target type of the goods to be estimated output by the clustering model, wherein the clustering model is trained in an unsupervised manner based on the sample fused data of each of the different types of second sample goods.

[0128] Based on any of the above embodiments, the target volume estimation model includes a convolution module, a pooling layer and a fully connected layer connected in sequence, and the convolution module includes multiple 3D convolution layers connected in sequence.

[0129] Figure 6 FIG is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention, such as Figure 6 As shown, the electronic device may include: a processor 610, a communications interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communications interface 620, and the memory 630 communicate with each other via the communication bus 640. The processor 610 may call logic instructions in the memory 630 to execute a cargo volume estimation method, which includes: acquiring multimodal data of the cargo to be estimated, the multimodal data including a structured light 3D point cloud of the cargo to be estimated, a radar point cloud of the cargo to be estimated, and an image of the cargo to be estimated;

[0130] performing registration and fusion processing on the structured light 3D point cloud, the radar point cloud, and the image to obtain fused data;

[0131] determining the target type of the cargo to be estimated based on the fused data;

[0132] The fused data is input into a target volume estimation model corresponding to the target type to obtain the volume of the cargo to be estimated output by the target volume estimation model, where the target volume estimation model is trained based on the sample fusion data of a first sample cargo corresponding to the target type.

[0133] Furthermore, the logic instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0134] On the other hand, the present invention further provides a computer program product, comprising a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the cargo volume estimation method provided by the above methods, the method comprising: acquiring multimodal data of cargo to be estimated, the multimodal data comprising a structured light 3D point cloud of the cargo to be estimated, a radar point cloud of the cargo to be estimated, and an image of the cargo to be estimated;

[0135] performing registration and fusion processing on the structured light 3D point cloud, the radar point cloud, and the image to obtain fused data;

[0136] determining the target type of the cargo to be estimated based on the fused data;

[0137] The fused data is input into a target volume estimation model corresponding to the target type to obtain the volume of the cargo to be estimated output by the target volume estimation model, where the target volume estimation model is trained based on the sample fusion data of a first sample cargo corresponding to the target type.

[0138] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the cargo volume estimation method provided by the above methods, the method comprising: acquiring multimodal data of cargo to be estimated, the multimodal data comprising a structured light 3D point cloud of the cargo to be estimated, a radar point cloud of the cargo to be estimated, and an image of the cargo to be estimated;

[0139] performing registration and fusion processing on the structured light 3D point cloud, the radar point cloud, and the image to obtain fused data;

[0140] determining the target type of the cargo to be estimated based on the fused data;

[0141] The fused data is input into a target volume estimation model corresponding to the target type to obtain the volume of the cargo to be estimated output by the target volume estimation model, where the target volume estimation model is trained based on the sample fusion data of a first sample cargo corresponding to the target type.

[0142] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0143] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A cargo volume estimation method, characterized in that: include: Acquire multimodal data of the goods to be estimated, the multimodal data including a structured light 3D point cloud of the goods to be estimated, a radar point cloud of the goods to be estimated, and an image of the goods to be estimated; performing registration and fusion processing on the structured light 3D point cloud, the radar point cloud, and the image to obtain fused data; determining the target type of the cargo to be estimated based on the fused data; Inputting the fused data into a target volume estimation model corresponding to the target type to obtain the volume of the cargo to be estimated output by the target volume estimation model; Inputting the fused data into a target volume estimation model corresponding to the target type to obtain the volume of the cargo to be estimated output by the target volume estimation model includes: Determining a target logistics scenario for the goods to be estimated; Inputting the fused data into a target volume estimation model corresponding to the target type and the target logistics scenario, and obtaining the volume of the cargo to be estimated output by the target volume estimation model; The target volume estimation model is trained based on the following method: Determine the second model structure parameters and the second model training parameters corresponding to the target type and the target logistics scenario based on the correspondence between the logistics scenario, type, model structure parameters and model training parameters; determining a second initial volume estimation model based on the second model structure parameters and the second model training parameters; Inputting the sample fusion data of the first sample cargo corresponding to the target type into the second initial volume estimation model, and obtaining a second predicted volume of the first sample cargo output by the second initial volume estimation model; Based on the second predicted volume and the volume label corresponding to the first sample cargo, the second model structure parameters and the second model training parameters are iteratively adjusted to obtain the target volume estimation model.

2. The cargo volume estimation method according to claim 1, characterized in that: The registering and fusing the structured light 3D point cloud, the radar point cloud, and the image to obtain fused data includes: Performing denoising on the structured light 3D point cloud and the radar point cloud based on a statistical filtering algorithm to obtain a denoised structured light 3D point cloud and a denoised radar point cloud; Performing denoising on the image based on a Gaussian filtering algorithm to obtain a denoised image; The denoised structured light 3D point cloud, the denoised radar point cloud and the denoised image are registered and fused to obtain the fused data.

3. The cargo volume estimation method according to claim 1, characterized in that: The determining the target type of the cargo to be estimated based on the fused data includes: The fused data is input into a clustering model to obtain the target type of the goods to be estimated output by the clustering model, wherein the clustering model is trained in an unsupervised manner based on the sample fused data of each of the different types of second sample goods.

4. The cargo volume estimation method according to claim 1, characterized in that: The target volume estimation model includes a convolution module, a pooling layer and a fully connected layer connected in sequence, and the convolution module includes multiple 3D convolution layers connected in sequence.

5. A cargo volume estimation device, characterized in that: include: an acquisition unit, configured to acquire multimodal data of the goods to be estimated, the multimodal data including a structured light 3D point cloud of the goods to be estimated, a radar point cloud of the goods to be estimated, and an image of the goods to be estimated; a registration and fusion unit, configured to perform registration and fusion processing on the structured light 3D point cloud, the radar point cloud, and the image to obtain fused data; a determining unit, configured to determine a target type of the cargo to be estimated based on the fused data; an estimating unit, configured to input the fused data into a target volume estimation model corresponding to the target type, and obtain the volume of the cargo to be estimated output by the target volume estimation model; The estimation unit is specifically configured to: Determining a target logistics scenario for the goods to be estimated; Inputting the fused data into a target volume estimation model corresponding to the target type and the target logistics scenario, and obtaining the volume of the cargo to be estimated output by the target volume estimation model; The target volume estimation model is trained based on the following method: Determine the second model structure parameters and the second model training parameters corresponding to the target type and the target logistics scenario based on the correspondence between the logistics scenario, type, model structure parameters and model training parameters; determining a second initial volume estimation model based on the second model structure parameters and the second model training parameters; Inputting the sample fusion data of the first sample cargo corresponding to the target type into the second initial volume estimation model, and obtaining a second predicted volume of the first sample cargo output by the second initial volume estimation model; Based on the second predicted volume and the volume label corresponding to the first sample cargo, the second model structure parameters and the second model training parameters are iteratively adjusted to obtain the target volume estimation model.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the cargo volume estimation method according to any one of claims 1 to 4 is implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the cargo volume estimation method according to any one of claims 1 to 4 is implemented.

Images