Method and apparatus for determining load rate based on neural network

Abstract

This application relates to a method and apparatus for determining loading rate based on a neural network. The method includes: acquiring an initial point cloud; filtering the initial point cloud according to the truck parking area and target height to obtain a preprocessed point cloud; using a preset neural network model to predict the vertex point cloud of the truck compartment in the preprocessed point cloud; processing the preprocessed point cloud according to the vertex point cloud to obtain a target point cloud, wherein the target point cloud includes the truck compartment and the point cloud inside the truck compartment; determining the cargo volume of the truck compartment using the target point cloud, and determining the loading rate of the truck compartment according to the cargo volume; and uploading vehicle loading information, including at least the loading rate, to a loading center platform. By processing the initial point cloud detected in real time by electromagnetic wave equipment through neural network prediction, the loading rate of the truck compartment can be accurately determined and synchronized to the loading center platform in real time.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method and apparatus for determining load rate based on neural networks. Background Technology

[0002] With the rapid development of the logistics sector, an increasing number of goods require transportation. When trucks transport goods, the loading center platform needs to determine whether to continue loading based on the truck's load factor, provided the truck is not overloaded. The load factor generally refers to the actual volume of goods carried divided by the maximum volume of goods the truck can carry.

[0003] In existing technologies, the loading volume and loading rate of traditional trucks are mainly estimated by the visual inspection and experience of loading personnel. This method cannot obtain the real-time loading volume and loading rate of the truck compartment, and the results are prone to significant errors. The loading center platform also cannot obtain the loading rate of the truck compartment in a timely manner, which is detrimental to the center's information processing and command. Summary of the Invention

[0004] In view of this, to address the technical problem that traditional methods rely mainly on the visual estimation and experience of loading personnel to determine the loading volume and loading rate of trucks, which cannot provide real-time loading volume and loading rate of the truck compartment and results in significant errors, and that the loading center platform cannot obtain the loading rate of the truck compartment in a timely manner, thus hindering the center's information processing and command, this application provides a method, apparatus, electronic device, and storage medium for determining loading rate based on neural networks.

[0005] In a first aspect, embodiments of this application provide a method for determining load rate based on a neural network, applied to an edge computing server, the method comprising:

[0006] Acquire an initial point cloud, wherein the initial point cloud includes a point cloud generated by an electromagnetic wave device located on the outer side of the rear of the carriage, which detects the area within the field of view of the electromagnetic wave device;

[0007] Based on the truck parking area and target height, the initial point cloud is filtered to obtain a preprocessed point cloud, wherein the target height is greater than or equal to the sum of the truck height and the first height of the electromagnetic wave device from the ground.

[0008] Using a preset neural network model, the vertex point cloud of the carriage in the preprocessed point cloud is predicted, wherein the vertex point cloud includes the preprocessed point cloud corresponding to the preset vertices of the carriage.

[0009] The preprocessed point cloud is processed based on the vertex point cloud to obtain the target point cloud, wherein the target point cloud includes the point cloud of the truck compartment and the interior of the truck compartment.

[0010] The cargo volume of the truck bed is determined using the target point cloud, and the loading rate of the truck bed is determined based on the cargo volume.

[0011] Upload vehicle loading information, including at least the aforementioned loading rate, to the loading center platform.

[0012] In one optional implementation, the preset neural network model is obtained in the following way:

[0013] Acquire an initial sample point cloud, wherein the initial sample point cloud includes a sample point cloud generated by an electromagnetic wave device probing a region within the field of view of the electromagnetic wave device;

[0014] Based on the truck parking area and target height, the initial sample point cloud is filtered to obtain the preprocessed sample point cloud;

[0015] The vertex sample point cloud of the carriage in the preprocessed sample point cloud is labeled, wherein the vertex sample point cloud includes the preprocessed sample point cloud corresponding to the preset vertices of the carriage;

[0016] Based on the labeled preprocessed sample point cloud, the preset neural network initial model is trained to obtain the preset neural network model;

[0017] The preset neural network initial model includes a point cloud classification network, the feature extraction network of the point cloud classification network is a backbone, the point cloud classification network includes multiple fully connected layers, and the output layer includes multiple neurons corresponding to vertex sample point clouds. The ReLU activation function is used between the output layer and the fully connected layer.

[0018] In an optional implementation, the step of processing the preprocessed point cloud based on the vertex point cloud to obtain the target point cloud includes:

[0019] Based on the vertex point cloud, a minimum bounding box is fitted, and the preprocessed point cloud located within the minimum bounding box is selected as the target point cloud from the preprocessed point cloud.

[0020] In an optional implementation, the step of processing the preprocessed point cloud based on the vertex point cloud to obtain the target point cloud includes:

[0021] Based on the vertex point cloud, determine the angle and first distance corresponding to the preprocessed point cloud;

[0022] The preprocessed point cloud is rotated by the angle around the second coordinate axis toward the first coordinate axis;

[0023] The preprocessed point cloud, after being rotated by the aforementioned angle, is translated along the negative direction of the third coordinate axis by the first distance.

[0024] Determine the second distance corresponding to the preprocessed point cloud after translating the first distance;

[0025] The preprocessed point cloud, after being translated by the first distance, is translated by the second distance along the negative direction of the first coordinate axis.

[0026] Determine the third distance corresponding to the preprocessed point cloud after translating the second distance;

[0027] The preprocessed point cloud, after being translated by the second distance, is translated by the third distance along the positive direction of the second coordinate axis to obtain the target point cloud.

[0028] In an optional implementation, determining the angle and first distance corresponding to the preprocessed point cloud based on the vertex point cloud includes:

[0029] By performing a planar projection on the vertex point cloud, a top view of the truck bed is obtained;

[0030] By performing line fitting on the top view, the first line equations corresponding to the first length and width boundaries of the truck body are obtained;

[0031] Based on the equation of the first straight line corresponding to the first long boundary on the right side of the truck body, the angle between the first long boundary on the right side and the first coordinate axis is determined, and the angle is determined to be the angle corresponding to the preprocessed point cloud.

[0032] Rotate the top view by the angle along the first coordinate axis to obtain the rotated top view;

[0033] By performing line fitting on the rotated top view, the second line equations corresponding to the second length and width boundaries of the truck body are obtained;

[0034] Based on the equation of the second straight line corresponding to the second long boundary on the right side of the truck body, the first distance between the second long boundary on the right side and the first coordinate axis is determined, and the first distance is determined to be the first distance corresponding to the preprocessed point cloud.

[0035] In an optional implementation, determining the second distance corresponding to the preprocessed point cloud after translating the first distance includes:

[0036] Based on the second straight line equation corresponding to the second wide boundary of the rear side of the truck body, determine the second distance between the second wide boundary of the rear side and the electromagnetic wave device;

[0037] The second distance is determined to be the second distance corresponding to the preprocessed point cloud after translating the first distance.

[0038] In an optional implementation, determining the third distance corresponding to the preprocessed point cloud after translating the second distance includes:

[0039] The preprocessed point cloud, after being translated by the second distance, is then projected onto a plane (YOZ plane) to obtain a front view of the truck bed.

[0040] By performing a straight line fit on the front view, the equations of the third straight line corresponding to the height and width boundaries of the truck body are obtained;

[0041] Based on the equation of the third straight line corresponding to the lower width boundary of the truck body, determine the second height between the lower width boundary and the third coordinate axis;

[0042] The difference between the first height and the second height of the electromagnetic wave device above the ground is obtained to obtain the third height, and the third height is determined to be the third distance corresponding to the preprocessed point cloud after translating the second distance.

[0043] In an optional implementation, determining the loading volume of the truck compartment using the target point cloud includes:

[0044] The target point cloud is downsampled using voxels to obtain the voxel-downsampled point cloud.

[0045] Based on the voxel downsampling point cloud, the actual width and actual height of the truck bed are obtained;

[0046] Construct an M*N empty matrix, where M includes the actual width and N includes the actual height;

[0047] According to the preset assignment rules, the target point cloud is assigned to the empty matrix;

[0048] Summing the empty matrix after assignment yields the loading volume of the truck bed.

[0049] In an optional implementation, assigning the target point cloud to the empty matrix according to a preset assignment rule includes:

[0050] Using the plane corresponding to the actual width and the actual height as a reference plane, for all target point clouds corresponding to any smallest unit in the reference plane, the target point cloud closest to the reference plane is selected and recorded in the corresponding position of the empty matrix corresponding to the smallest unit.

[0051] Secondly, embodiments of this application provide a load rate determination device based on a neural network, applied to an edge computing server, the device comprising:

[0052] An initial point cloud acquisition module is used to acquire an initial point cloud, wherein the initial point cloud includes a point cloud generated by an electromagnetic wave device located on the outer side of the rear of the carriage detecting the area within the field of view of the electromagnetic wave device;

[0053] The initial point cloud filtering module is used to filter the initial point cloud according to the truck parking area and the target height to obtain a preprocessed point cloud, wherein the target height is greater than or equal to the sum of the truck height and the first height of the electromagnetic wave device from the ground.

[0054] The vertex point cloud prediction module is used to predict the vertex point cloud of the carriage in the preprocessed point cloud using a preset neural network model, wherein the vertex point cloud includes the preprocessed point cloud corresponding to the preset vertices of the carriage.

[0055] A vertex point cloud processing module is used to process the preprocessed point cloud based on the vertex point cloud to obtain a target point cloud, wherein the target point cloud includes the point cloud of the truck car and the interior of the truck car.

[0056] The loading rate determination module is used to determine the cargo volume of the truck compartment using the target point cloud, and to determine the loading rate of the truck compartment based on the cargo volume.

[0057] The loading information upload module is used to upload vehicle loading information, including at least the loading rate, to the loading center platform.

[0058] Thirdly, embodiments of this application provide an electronic device, including: a processor and a memory, wherein the processor is configured to execute a neural network-based load rate determination program stored in the memory to implement any of the neural network-based load rate determination methods in the first aspect.

[0059] Fourthly, embodiments of this application provide a storage medium storing one or more programs, which can be executed by one or more processors to implement any of the neural network-based load rate determination methods in the first aspect.

[0060] The technical solution provided in this application embodiment obtains an initial point cloud, wherein the initial point cloud includes a point cloud generated by an electromagnetic wave device located on the outer side of the rear of the truck body, detecting the area within the field of view of the electromagnetic wave device; the initial point cloud is filtered according to the truck parking area and the target height to obtain a preprocessed point cloud, wherein the target height is greater than or equal to the sum of the truck height and the first height of the electromagnetic wave device from the ground; a preset neural network model is used to predict the vertex point cloud of the truck body in the preprocessed point cloud, wherein the vertex point cloud includes the preprocessed point cloud corresponding to preset vertices of the truck body; the preprocessed point cloud is processed according to the vertex point cloud to obtain a target point cloud, wherein the target point cloud includes the point cloud of the truck body and the interior of the truck body; the cargo volume of the truck body is determined using the target point cloud, and the loading rate of the truck body is determined according to the cargo volume; vehicle loading information including at least the loading rate is uploaded to the loading center platform. Therefore, it is possible to process the initial point cloud of electromagnetic wave equipment in real time through neural network prediction, more accurately determine the loading rate of the truck compartment, and synchronize the loading rate to the loading center platform in real time. This facilitates information processing and command of the loading center platform, enables the scheduling of trucks, and greatly improves the utilization rate of trucks. Attached Figure Description

[0061] Figure 1 A top view of a truck when parked, provided as an embodiment of this application;

[0062] Figure 2 A side view of a truck in actual parking state provided for an embodiment of this application;

[0063] Figure 3 A front view of a truck in its actual parking state, provided as an embodiment of this application;

[0064] Figure 4 A schematic diagram illustrating the implementation process of a neural network-based method for determining load rate, provided in an embodiment of this application;

[0065] Figure 5 Another top view of a truck parked as provided in this application embodiment;

[0066] Figure 6 A top view of a target point cloud provided in an embodiment of this application;

[0067] Figure 7 A schematic diagram of a target point cloud provided for an embodiment of this application;

[0068] Figure 8 A schematic diagram illustrating the implementation process of a neural network model training method provided in this application embodiment;

[0069] Figure 9A schematic diagram illustrating the implementation process of another neural network-based method for determining load rate provided in this application embodiment;

[0070] Figure 10 A schematic diagram illustrating the implementation process of another neural network-based method for determining load rate provided in this application embodiment;

[0071] Figure 11 A schematic diagram illustrating the implementation process of a first distance determination method provided in this application embodiment;

[0072] Figure 12 A schematic diagram illustrating the implementation process of a second distance determination method provided in this application embodiment;

[0073] Figure 13 A schematic diagram illustrating the implementation process of a third distance determination method provided in this application embodiment;

[0074] Figure 14 A schematic diagram illustrating the implementation process of another neural network-based method for determining load rate provided in this application embodiment;

[0075] Figure 15 A schematic diagram of an empty matrix provided in an embodiment of this application;

[0076] Figure 16 A schematic diagram of a matrix after assignment provided in an embodiment of this application;

[0077] Figure 17 A schematic diagram illustrating the implementation process of a neural network-based method for determining load rate, provided for an embodiment of this application;

[0078] Figure 18 A schematic diagram of a load rate determination device based on a neural network provided in an embodiment of this application;

[0079] Figure 19 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0080] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0081] In this embodiment, the specifications of the truck to be loaded are pre-defined: vehicle length: 4.2-17 meters, vehicle width: 1.9-2.5 meters, vehicle height (height of the truck bed roof from the ground): 1.8-2.7 meters. A rectangular area is pre-defined as the truck parking area, with a length of 18 meters and a width of 5 meters. The electromagnetic wave device is installed on one side of the wider side of the rectangular area, at a distance of 1 meter or more from the wider side, and at a height of 2 meters or more from the ground. Figure 1 As shown, Figure 1 This is a top view of a truck parked according to an embodiment of this application. The outer rectangle is a rectangular area, the inner rectangle is the truck parked inside the rectangular area, the dots outside the outer rectangle are electromagnetic wave devices, the distance L between the electromagnetic wave devices and the outer rectangle is greater than or equal to 1 meter, and the distance between the electromagnetic wave devices and the ground is greater than or equal to 2 meters.

[0082] In this embodiment, the cargo inside the truck bed is stacked sequentially from the front of the truck to the rear. When the truck is parked within the rectangular area, in order for the electromagnetic wave equipment to clearly detect the internal point cloud of the truck, the rear of the truck needs to face the side where the electromagnetic wave equipment is located. The actual truck parking and rectangular position can be found in [reference needed]. Figure 2 , Figure 3 , Figure 2 This application provides a side view of a truck in its actual parking state, as shown in an embodiment. Figure 3 This is a front view of a truck in its actual parking state, provided as an embodiment of this application.

[0083] It should be noted that in the embodiments of this application, the description of the left and right sides of the truck bed refers to the left and right sides as observed from the outside of the rear of the truck bed, and will not be repeated below.

[0084] Figure 4 This application provides a schematic diagram of the implementation process of a neural network-based load rate determination method, applied to an edge computing server. The method may include the following steps:

[0085] S401: Acquire initial point cloud, wherein the initial point cloud includes point cloud generated by the electromagnetic wave device located on the outer side of the rear of the carriage, which detects the area within the field of view of the electromagnetic wave device.

[0086] In this embodiment, a point cloud generated by an electromagnetic wave device located on the outer side of the rear of the carriage, probing the area within its field of view, is first acquired; this point cloud is referred to as the initial point cloud. The electromagnetic wave device first collects raw UDP (User Datagram Protocol) data packets of the point cloud data within its field of view, and then parses and calculates the initial point cloud according to the device's specifications and product manual. The electromagnetic wave device can be a linear array lidar, and this application does not limit its use. When the electromagnetic wave device is a linear array lidar, the raw UDP data packets of the point cloud data within the lidar's field of view are collected, and then the initial point cloud is calculated and parsed according to the lidar's specifications and product manual.

[0087] S402: Based on the truck parking area and target height, the initial point cloud is filtered to obtain a preprocessed point cloud, wherein the target height is greater than or equal to the sum of the truck height and the first height of the electromagnetic wave device from the ground.

[0088] In this embodiment, the initial point cloud is filtered, and all point clouds outside the aforementioned rectangular region and target height range are cropped and removed to obtain a preprocessed point cloud. The target height can be set to be greater than or equal to the sum of the truck's height and the electromagnetic wave device's initial height above the ground. The truck's height can be its actual height or its maximum height; this application does not limit this. For example, if the truck's actual height is 2 meters and the electromagnetic wave device's initial height above the ground is 2 meters, then the target height is greater than or equal to 4 meters.

[0089] S403: Using a preset neural network model, predict the vertex point cloud of the carriage in the preprocessed point cloud, wherein the vertex point cloud includes the preprocessed point cloud corresponding to the preset vertices of the carriage.

[0090] In this embodiment, a preset neural network model is used to predict the vertex point cloud of the carriage in the preprocessed point cloud. The vertex point cloud includes the preprocessed point cloud corresponding to preset vertices of the carriage. Specifically, the preset vertices include: the lower left rear, lower right rear, lower right front, and upper right rear of the freight car carriage.

[0091] S404: Process the preprocessed point cloud based on the vertex point cloud to obtain the target point cloud, wherein the target point cloud includes the point cloud of the freight car and the interior of the freight car.

[0092] In this embodiment of the application, the preprocessed point cloud is processed according to the vertex point cloud to obtain the target point cloud, wherein the target point cloud includes the point cloud of the freight car and the interior of the freight car.

[0093] In this embodiment of the application, the origin of the target point cloud in three-dimensional space is the lower right corner vertex of the truck bed, the bottom surface of the truck bed is the XOY plane of three-dimensional space, and the right plane of the truck bed is the XOZ plane of three-dimensional space. That is, the point cloud along the three sides of the lower right corner of the rear of the truck bed as a three-dimensional orthogonal basis is the target point cloud. The positive x-axis is the direction from the rear of the truck bed to the front of the truck bed, the positive y-axis is the direction from the right side of the truck bed to the left side, and the positive z-axis is the direction perpendicular to the horizontal plane and upward.

[0094] In this embodiment, the plane containing the electromagnetic wave device, parallel to the rectangular region, is taken as the XOY plane. A straight line parallel to the right long boundary of the rectangular region and located directly above the right long boundary is taken as the x-axis. A straight line parallel to the rear wide boundary of the rectangular region and containing the electromagnetic wave device is taken as the y-axis. Thus, the z-axis is obtained as a straight line perpendicular to the rectangular region and passing through the intersection of the x-axis and y-axis. Figure 5 As shown, Figure 5 This application provides another top view of a truck parked in an embodiment of the present application. Figure 5 The positive directions of the x-axis and y-axis are marked, from which the positive direction of the z-axis can be obtained, and from the above, the height of the origin can be obtained as the distance between the electromagnetic wave device and the ground.

[0095] It should be noted that the embodiments of this application only provide an example of a coordinate system origin. The coordinate system origin can also be set to other locations. For example, the coordinate system origin can also be set at the location of the electromagnetic wave device. The directions of each coordinate axis are the same as above, and this application does not limit this. Therefore, the intermediate results calculated using the method provided in this application may differ, but the final determined loading rate of the truck compartment will be the same.

[0096] In the real-time example of this application, the preprocessed point cloud needs to be processed based on the vertex point cloud to ensure that the lower right corner vertex of the carriage coincides with the origin of the coordinate system, and that the point cloud of the carriage and its interior is in the state of the target point cloud. A top view of the target point cloud is shown below. Figure 6 As shown, Figure 6 This application provides a top view of a target point cloud, as shown in the embodiment of the present application. The actual target point cloud is as follows: Figure 7 As shown, Figure 7 This is a schematic diagram of a target point cloud provided in an embodiment of this application. From left to right, it is the front view (observed from the rear of the truck), top view, and left view of the actual target point cloud.

[0097] S405: Use the target point cloud to determine the cargo volume of the truck bed, and determine the loading rate of the truck bed based on the cargo volume.

[0098] In this embodiment of the application, after obtaining the target point cloud, the loading volume of the truck compartment can be determined using the target point cloud, and then the loading rate of the truck compartment can be determined based on the loading volume. Specifically, the loading rate of the truck compartment is determined by calculating the ratio of the loading volume to the total volume of the truck compartment.

[0099] S406: Upload vehicle loading information, including at least the loading rate, to the loading center platform.

[0100] In this embodiment, after obtaining the loading rate of the truck bed, vehicle loading information, including at least the loading rate, is uploaded to the transfer center platform. The loading center platform can process the real-time received truck information and schedule trucks according to the loading information, thereby improving the utilization rate of the trucks. The loading information includes the vehicle loading rate, and may also include truck bed specifications, remaining fuel level, etc., which are not limited in this application.

[0101] Based on the above description of the technical solution provided in the embodiments of this application, this application uses an electromagnetic wave device to acquire an initial point cloud, filters the initial point cloud to obtain a preprocessed point cloud, uses a preset neural network model to predict the vertex point cloud of the truck compartment in the preprocessed point cloud, processes the preprocessed point cloud based on the vertex point cloud to obtain a target point cloud, determines the cargo volume of the truck compartment through the target point cloud, and determines the loading rate of the truck compartment based on the cargo volume. First, the electromagnetic wave device can acquire the initial point cloud of the truck compartment in real time, realizing accurate determination of the loading rate of the compartment during loading and unloading. This reduces the error of the loading rate on the one hand, and can synchronize the loading rate of the truck compartment to the loading center platform in real time. The loading center platform acquires the real-time loading rate of the truck compartment and schedules trucks according to the loading rate of the truck compartment to improve the utilization rate of the trucks. Second, the use of a preset neural network in the process of processing the initial point cloud to obtain the target point cloud makes the acquisition of the target point cloud faster.

[0102] Specifically, the neural network model in S403 uses... Figure 8 The method shown is used to obtain, Figure 8 This is a schematic diagram illustrating the implementation process of a neural network model training method provided in an embodiment of this application. The method may include the following steps:

[0103] S801: Acquire initial sample point cloud, wherein the initial sample point cloud includes sample point cloud generated by the electromagnetic wave device probing the area within the field of view of the electromagnetic wave device.

[0104] In this embodiment, a preset number of sample point clouds are obtained. The preset number can be 1000, 2000, etc., and this application does not limit this. The method for obtaining the initial sample point cloud is the same as the method for obtaining the initial point cloud in S401 above, and will not be repeated here.

[0105] S802: Based on the truck parking area and target height, the initial sample point cloud is filtered to obtain the preprocessed sample point cloud.

[0106] In this embodiment of the application, the method for filtering the initial sample point cloud to obtain the preprocessed sample point cloud is the same as the method for filtering the initial point cloud to obtain the preprocessed point cloud in S402, and will not be described again here.

[0107] S803: Annotate the vertex sample point cloud of the carriage in the preprocessed sample point cloud, wherein the vertex sample point cloud includes the preprocessed sample point cloud corresponding to the preset vertices of the carriage.

[0108] In this embodiment, the vertex sample point cloud of the carriage in the preprocessed sample point cloud is labeled to obtain the preprocessed sample point cloud corresponding to the preset vertices of the labeled carriage, denoted as the vertex sample point cloud. The preset vertices include: the lower left rear, lower right rear, lower right front, and upper right rear of the freight car, a total of four vertices. Therefore, a sample includes the preprocessed point cloud and the coordinates of the four manually labeled vertices.

[0109] S804: Based on the labeled preprocessed sample point cloud, the preset neural network initial model is trained to obtain the preset neural network model. The preset neural network initial model includes a point cloud classification network, the feature extraction network of the point cloud classification network is the backbone, the point cloud classification network includes multiple fully connected layers, and the output layer includes multiple neurons corresponding to the vertex sample point cloud. The ReLU activation function is used between the output layer and the fully connected layer.

[0110] In this embodiment, a preset neural network initial model is trained based on the labeled preprocessed sample point cloud to obtain the preset neural network model. The preset neural network initial model includes a point cloud classification network, the feature extraction network of the point cloud classification network is a backbone, the point cloud classification network includes multiple fully connected layers, and the output layer includes multiple neurons corresponding to the vertex sample point cloud. The ReLU activation function is used between the output layer and the fully connected layer.

[0111] For example, a point cloud classification network adds 3-4 fully connected layers, with the final output layer containing 4*3 neurons, each corresponding to one of the four predicted vertex coordinates. The ReLU activation function is used between the output layer and the fully connected layers. When training the initial model of the preset neural network using labeled preprocessed sample point clouds, an L1+L2 loss function is used, and the Adam optimizer is employed.

[0112] Based on the above description of the technical solutions provided in the embodiments of this application, the point cloud classification network constructed in this application includes multiple connection layers, and the output layer includes multiple neurons corresponding to the vertex sample point cloud, enabling the trained preset neural network model to more accurately predict the vertex point cloud. In this application, labeled sample point clouds are used to train the constructed preset neural network initial model to obtain the preset neural network model. Then, when the loading rate is determined, the preset neural network model can be used to accurately predict the vertex point cloud of the carriage in the preprocessed point cloud.

[0113] Figure 9 A schematic diagram illustrating the implementation process of another neural network-based load rate determination method provided in this application embodiment, the method may include the following steps:

[0114] S901: Acquire initial point cloud, wherein the initial point cloud includes point cloud generated by the electromagnetic wave device located on the outer side of the rear of the carriage, which detects the area within the field of view of the electromagnetic wave device.

[0115] S902: Based on the truck parking area and target height, the initial point cloud is filtered to obtain a preprocessed point cloud, wherein the target height is greater than or equal to the sum of the truck height and the first height of the electromagnetic wave device from the ground.

[0116] S903: Using a preset neural network model, predict the vertex point cloud of the carriage in the preprocessed point cloud, wherein the vertex point cloud includes the preprocessed point cloud corresponding to the preset vertices of the carriage.

[0117] In the embodiments of this application, S901 to S903 have been described in detail in S401 to S403, and will not be repeated here.

[0118] S904: Based on the vertex point cloud, fit the minimum bounding box, and select the preprocessed point cloud that is located within the minimum bounding box as the target point cloud.

[0119] In this embodiment, a minimum bounding cuboid box with four vertex point clouds as its four vertices is fitted. Since the four vertices are the four vertices of the carriage, and can be connected and completed based on these four vertices to obtain a cuboid, this cuboid is the carriage, and this cuboid is regarded as the minimum bounding cuboid box. That is, the minimum bounding cuboid box is the minimum bounding cuboid box that can enclose the freight car carriage.

[0120] S905: Use the target point cloud to determine the cargo volume of the truck bed, and determine the loading rate of the truck bed based on the cargo volume.

[0121] S906: Upload vehicle loading information, including at least the loading rate, to the loading center platform.

[0122] In the embodiments of this application, S905 and S906 have been described in detail in S405 and S406, and will not be repeated here.

[0123] Based on the above description of the technical solutions provided in the embodiments of this application, this application uses vertex point clouds to fit a minimum bounding cuboid box, and selects the preprocessed point cloud located within the minimum bounding cuboid box as the target point cloud. The minimum bounding cuboid box can be determined using the vertex point cloud, making the fitted minimum bounding cuboid box more accurately coincide with the carriage.

[0124] Figure 10 A schematic diagram illustrating the implementation process of another neural network-based load rate determination method provided in this application embodiment, the method may include the following steps:

[0125] S1001: Obtain the initial point cloud, wherein the initial point cloud includes the point cloud generated by the electromagnetic wave device located on the outer side of the rear of the carriage, which detects the area within the field of view of the electromagnetic wave device.

[0126] S1002: Based on the truck parking area and target height, the initial point cloud is filtered to obtain a preprocessed point cloud, wherein the target height is greater than or equal to the sum of the truck height and the first height of the electromagnetic wave device from the ground.

[0127] S1003: Using a preset neural network model, predict the vertex point cloud of the carriage in the preprocessed point cloud, wherein the vertex point cloud includes the preprocessed point cloud corresponding to the preset vertices of the carriage.

[0128] In the embodiments of this application, S1001 to S1003 have been described in detail in S401 to S403, and will not be repeated here.

[0129] S1004: Based on the vertex point cloud, determine the angle and first distance corresponding to the preprocessed point cloud.

[0130] In this embodiment of the application, the angle and first distance corresponding to the preprocessed point cloud are determined based on the vertex point cloud. Specifically, the method for determining the angle and first distance corresponding to the preprocessed point cloud is as follows: Figure 11 As shown, Figure 11 This application provides a schematic diagram of the implementation process of a first distance determination method, which may include the following steps:

[0131] S1101: Project the vertex point cloud onto a plane to obtain a top view of the truck bed.

[0132] In this embodiment, the cuboid fitted from the vertex point cloud represents the truck bed. Therefore, the cuboid fitted from the vertex point cloud and the top view obtained by projecting the cuboid onto a plane are the top views of the truck bed. For example, projecting the vertex point cloud onto the XOY plane yields the top view of the truck bed.

[0133] S1102: Perform line fitting on the top view to obtain the first line equations corresponding to the first length and width boundaries of the truck body.

[0134] In this embodiment of the application, by performing line fitting on the top view, the line equations of each boundary of the truck body in the XOY plane can be obtained, and the obtained line equations are denoted as the first line equations corresponding to the first length and width boundaries of the truck body.

[0135] S1103: Based on the equation of the first straight line corresponding to the first long boundary on the right side of the truck body, determine the angle between the first long boundary on the right side and the first coordinate axis, and determine the angle as the angle corresponding to the preprocessed point cloud.

[0136] In this embodiment, the angle between the first long boundary on the right side of the truck body and the first coordinate axis is determined based on the equation of the first straight line corresponding to the first long boundary on the right side of the truck body. The angle is determined to be the angle corresponding to the preprocessed point cloud. The first coordinate axis is the x-axis.

[0137] S1104: Rotate the top view by the above angle toward the first coordinate axis to obtain the rotated top view.

[0138] In this embodiment, the top view is rotated by the aforementioned angle toward the first coordinate axis to obtain a rotated top view, at which point the long boundary of the top view is parallel to the first coordinate axis. For example, the aforementioned angle is 20°, and the top view is rotated 20° counterclockwise or clockwise depending on the actual situation. It should be noted that the operation of rotating the top view toward the first coordinate axis by the aforementioned angle is simulated by an edge computing server, the purpose of which is to obtain the first distance between the long boundary of the top view and the first coordinate axis.

[0139] In one possible implementation, S1005 can be executed after S1103 is executed to determine the angle corresponding to the preprocessed point cloud, and then S1105 and S1106 can be executed to determine the first distance corresponding to the preprocessed point cloud. This application does not limit this.

[0140] S1105: Perform line fitting on the rotated top view to obtain the second line equations corresponding to the second length and width boundaries of the truck body.

[0141] In this embodiment of the application, by performing line fitting on the rotated top view, the straight line equations of each boundary of the rotated truck body in the XOY plane can be obtained. The straight line equations obtained at this time are denoted as the second straight line equations corresponding to the second length and width boundaries of the truck body.

[0142] S1106: Based on the equation of the second straight line corresponding to the second long boundary on the right side of the truck body, determine the first distance between the second long boundary on the right side and the first coordinate axis, and determine the first distance as the first distance corresponding to the preprocessed point cloud.

[0143] In this embodiment, a first distance between the second long boundary on the right side of the truck bed and the first coordinate axis is determined based on the equation of the second straight line corresponding to the second long boundary on the right side of the truck bed, and this first distance is determined to be the first distance corresponding to the preprocessed point cloud. Here, the first coordinate axis is the x-axis, and the second long boundary on the right side of the truck bed is the side of the truck bed's long boundary that is closest to the first coordinate axis. For example, if the distance between the equation of the second straight line corresponding to the second long boundary on the right side of the truck bed and the first coordinate axis is 20 centimeters, then the first distance is determined to be 20 centimeters.

[0144] S1005: Rotate the preprocessed point cloud around the second coordinate axis toward the first coordinate axis by the aforementioned angle.

[0145] In this embodiment, the second coordinate axis is the z-axis. The preprocessed point cloud is rotated around the second coordinate axis towards the first coordinate axis by the aforementioned angle. The method for determining the aforementioned angle is described in [the following text is missing from the original] Figure 11 The details are provided in the document, so I will not repeat them here.

[0146] S1006: Translate the preprocessed point cloud after rotation by a first distance along the negative direction of the third coordinate axis.

[0147] In this embodiment, the third coordinate axis is the y-axis. After determining the first distance, the preprocessed point cloud after rotation is translated by the first distance along the negative direction of the third coordinate axis. For example, if the first distance is 20 centimeters, the outlier removal point cloud is translated by 20 centimeters along the negative direction of the third coordinate axis.

[0148] S1007: Determine the second distance corresponding to the preprocessed point cloud after translating the first distance.

[0149] In this embodiment, a second distance is determined corresponding to the preprocessed point cloud after translation by a first distance. The second distance is the distance between the wide boundary of the preprocessed point cloud after translation by the first distance and the electromagnetic wave device. It can be understood that the straight line where the electromagnetic wave device is located is the third coordinate axis in the coordinate system, namely the y-axis.

[0150] Specifically, the methods for determining the angle and second distance corresponding to the preprocessed point cloud are as follows: Figure 12 As shown, Figure 12This is a schematic diagram illustrating the implementation process of a second distance determination method provided in an embodiment of this application. The method may include the following steps:

[0151] S1201: Project the vertex point cloud onto a plane to obtain a top view of the truck bed.

[0152] S1202: Perform line fitting on the top view to obtain the first line equations corresponding to the first length and width boundaries of the truck body.

[0153] S1203: Based on the equation of the first straight line corresponding to the first long boundary on the right side of the truck body, determine the angle between the first long boundary on the right side and the first coordinate axis, and determine the angle as the angle corresponding to the preprocessed point cloud.

[0154] S1204: Rotate the top view by an angle toward the first coordinate axis to obtain the rotated top view.

[0155] S1205: Perform line fitting on the rotated top view to obtain the second line equations corresponding to the second length and width boundaries of the truck body.

[0156] In the embodiments of this application, S1201 to S1205 have been described in detail in S1101 to S1105, and will not be repeated here.

[0157] S1206: Determine the second distance between the second wide boundary at the rear of the truck body and the electromagnetic wave device based on the second straight line equation corresponding to the second wide boundary at the rear of the truck body.

[0158] S1207: Determine the second distance as the second distance corresponding to the preprocessed point cloud after translating the first distance.

[0159] In this embodiment, a second distance is determined between the second wide boundary at the rear of the truck bed and the electromagnetic wave device based on the second straight line equation corresponding to the second wide boundary at the rear of the truck bed. This second distance is then determined to be the second distance corresponding to the preprocessed point cloud after translating the first distance. Specifically, if the second straight line equation corresponding to the second wide boundary at the rear of the truck bed and the electromagnetic wave device is 2 meters, then the second distance is determined to be 2 meters.

[0160] S1008: Translate the preprocessed point cloud, after being translated by the first distance, by the second distance along the negative direction of the first coordinate axis.

[0161] In this embodiment, the first coordinate axis is the x-axis. After determining the second distance, the preprocessed point cloud, after being translated by the first distance, is translated by the second distance along the negative direction of the first coordinate axis. For example, if the second distance is 2 meters, then the outlier removal point cloud is translated by 2 meters along the negative direction of the first coordinate axis.

[0162] S1009: Determine the third distance corresponding to the preprocessed point cloud after translating the second distance.

[0163] In this embodiment, a third distance is determined corresponding to the preprocessed point cloud after translation by a second distance. Specifically, the method for determining the angle corresponding to the preprocessed point cloud and the third distance is as follows: Figure 13 As shown, Figure 13 This application provides a schematic diagram of the implementation process of a third distance determination method, which may include the following steps:

[0164] S1301: Project the preprocessed point cloud after translation by the second distance onto a plane to obtain the front view of the freight car.

[0165] In this embodiment, the preprocessed point cloud, after being translated by a second distance, is projected onto a plane to obtain a front view of the truck bed. For example, the preprocessed point cloud is projected onto the YOZ plane to obtain a front view of the truck bed. It is understood that the rear side of the truck bed in the preprocessed point cloud after being translated by a second distance coincides with the YOZ plane.

[0166] S1302: Perform line fitting on the front view to obtain the third line equations corresponding to the height and width boundaries of the truck body.

[0167] In this embodiment of the application, a straight line is fitted to the front view, and the resulting straight line equation is denoted as the third straight line equation corresponding to the height and width boundaries of the truck body.

[0168] S1303: Determine the second height between the lower width boundary and the third coordinate axis based on the equation of the third straight line corresponding to the lower width boundary of the truck body.

[0169] In this embodiment of the application, the second height between the lower width boundary and the third coordinate axis is determined based on the equation of the third straight line corresponding to the lower width boundary of the truck body. The third coordinate axis is the y-axis.

[0170] S1304: Obtain the difference between the first and second heights of the electromagnetic wave device above the ground to obtain the third height, and determine that the third height is the third distance corresponding to the preprocessed point cloud after translating the second distance.

[0171] In this embodiment, the difference between the first and second heights of the electromagnetic wave device above the ground is obtained to determine the third height. This third height is then determined to be the third distance corresponding to the preprocessed point cloud after translation by the second distance. For example, if the first height is 2 meters and the second height is 1 meter, then the third height is 1 meter.

[0172] S1010: Translate the preprocessed point cloud, which has been translated by the second distance, by the third distance along the positive direction of the second coordinate axis to obtain the target point cloud.

[0173] In this embodiment of the application, the second coordinate axis is the z-axis. The preprocessed point cloud after being translated by a second distance is translated by a third distance along the positive direction of the second coordinate axis. At this time, the electromagnetic wave device is located on the lower wide boundary of the truck bed, that is, the lower wide boundary of the truck bed coincides with the second coordinate axis. The point cloud outside the truck bed is cropped according to the vertex point cloud. The point cloud at this time is the target point cloud.

[0174] S1011: Use the target point cloud to determine the cargo volume of the truck bed, and determine the loading rate of the truck bed based on the cargo volume.

[0175] S1012: Upload vehicle loading information, including at least the loading rate, to the loading center platform.

[0176] In the embodiments of this application, S1011 and S1012 have been described in detail in S405 and S406, and will not be repeated here.

[0177] Figure 14 A schematic diagram illustrating the implementation process of another neural network-based load rate determination method provided in this application embodiment, the method may include the following steps:

[0178] S1401: Obtain the initial point cloud, wherein the initial point cloud includes the point cloud generated by the electromagnetic wave device located on the outer side of the rear of the carriage, which detects the area within the field of view of the electromagnetic wave device.

[0179] S1402: Based on the truck parking area and target height, the initial point cloud is filtered to obtain a preprocessed point cloud, wherein the target height is greater than or equal to the sum of the truck height and the first height of the electromagnetic wave device from the ground.

[0180] S1403: Using a preset neural network model, predict the vertex point cloud of the carriage in the preprocessed point cloud, wherein the vertex point cloud includes the preprocessed point cloud corresponding to the preset vertices of the carriage.

[0181] S1404: Process the preprocessed point cloud based on the vertex point cloud to obtain the target point cloud, wherein the target point cloud includes the point cloud of the freight car and the interior of the freight car.

[0182] In the embodiments of this application, S1401 to S1404 have been described in detail in S401 to S404, and will not be repeated here.

[0183] S1405: Perform voxel downsampling on the target point cloud to obtain the voxel downsampled point cloud.

[0184] S1406: Obtain the actual width and height of the truck bed based on the point cloud after voxel downsampling.

[0185] S1407: Construct an M*N empty matrix, where M includes the actual width and N includes the actual height.

[0186] The following provides a unified explanation of S1405 to S1407:

[0187] In this embodiment of the application, after obtaining the target point cloud, voxel downsampling is performed on the target point cloud to obtain a voxel downsampled point cloud. For example, using 1 cm as the voxel downsampling unit size, the target point cloud is downsampled to obtain a voxel downsampled point cloud.

[0188] In this embodiment, the actual width and height of the truck bed are obtained from the voxel-downsampled point cloud. Specifically, the number of voxel boxes included in the width and height of the voxel-downsampled point cloud can be determined. The actual width M and actual height N of the truck bed are determined based on the dimensions of the voxel boxes. Using M and N as dimensions, an empty matrix of dimensions M*N is constructed. Figure 15 As shown, Figure 15 This is a schematic diagram of an empty matrix provided in an embodiment of this application. Figure 15 The horizontal axis represents the width of the vehicle, and the vertical axis represents its height.

[0189] S1408: Assign the target point cloud to an empty matrix according to the preset assignment rules.

[0190] In this embodiment of the application, the target point cloud is assigned to the aforementioned empty matrix according to a preset assignment rule, and the assigned matrix is ​​as follows: Figure 16 As shown, Figure 16 This is a schematic diagram of a matrix after assignment, provided as an embodiment of this application.

[0191] S1409: Summing the empty matrix after assignment yields the loading volume of the freight car.

[0192] In this embodiment of the application, the summation of the empty matrix after assignment is recorded as the loading volume of the truck compartment.

[0193] S1410: Determine the loading rate of the freight car body based on the loading volume.

[0194] S1411: Upload vehicle loading information, including at least the loading rate, to the loading center platform.

[0195] In the embodiments of this application, S1410 and S1411 have been described in detail in S405 and S406, and will not be repeated here.

[0196] Figure 17 A schematic diagram illustrating the implementation process of a neural network-based load rate determination method provided in this application embodiment, the method including the following steps:

[0197] S1701: Acquire initial point cloud, wherein the initial point cloud includes point cloud generated by the electromagnetic wave device located on the outer side of the rear of the carriage, which detects the area within the field of view of the electromagnetic wave device.

[0198] S1702: Based on the truck parking area and target height, the initial point cloud is filtered to obtain a preprocessed point cloud, wherein the target height is greater than or equal to the sum of the truck height and the first height of the electromagnetic wave device from the ground.

[0199] S1703: Using a preset neural network model, predict the vertex point cloud of the carriage in the preprocessed point cloud, wherein the vertex point cloud includes the preprocessed point cloud corresponding to the preset vertices of the carriage.

[0200] S1704: Process the preprocessed point cloud based on the vertex point cloud to obtain the target point cloud, wherein the target point cloud includes the point cloud of the freight car and the interior of the freight car.

[0201] S1705: Perform voxel downsampling on the target point cloud to obtain the voxel downsampled point cloud.

[0202] S1706: Obtain the actual width and height of the truck bed based on the point cloud after voxel downsampling.

[0203] S1707: Construct an M*N empty matrix, where M includes the actual width and N includes the actual height.

[0204] In the embodiments of this application, S1701 to S1707 have been described in detail in S1401 to S1407, and will not be repeated here.

[0205] S1708: Using the plane corresponding to the actual width and actual height as the reference plane, for all target point clouds corresponding to any smallest unit in the reference plane, select the target point cloud closest to the reference plane and record it in the corresponding position in the empty matrix corresponding to the smallest unit.

[0206] In this embodiment, the plane corresponding to the actual width and actual height is used as the reference plane. For all target point clouds corresponding to the smallest unit in the reference plane, the target point cloud closest to the reference plane is selected and recorded in the corresponding position of the empty matrix corresponding to the smallest unit.

[0207] In this embodiment, since the goods are stacked sequentially from the front of the truck to the rear, a reference plane is set at the truck bed boundary in the rear direction. The smallest unit in the reference plane includes a square with a voxel box size of [missing information - likely a side length]. For any smallest unit, the target point cloud closest to the reference plane among all target point clouds corresponding to that unit is determined. The distance between the target point cloud closest to the reference plane and the truck bed boundary in the front direction is recorded in the corresponding position of the empty matrix corresponding to the smallest unit. Specifically, the number of voxel boxes between the target point cloud closest to the reference plane and the truck bed boundary in the front direction can be determined first. The product of the number of voxel boxes and the volume of one voxel box is used as the distance between the target point cloud closest to the reference plane and the truck bed boundary in the front direction, and this distance is recorded in the corresponding position of the empty matrix corresponding to the smallest unit.

[0208] S1709: Summing the empty matrix after assignment yields the loading volume of the freight car.

[0209] S1710: Determine the loading rate of the freight car body based on the loading volume.

[0210] S1711: Upload vehicle loading information, including at least the loading rate, to the loading center platform.

[0211] In the embodiments of this application, S1709 to S711 have been described in detail in S1409 to S1411, and will not be repeated here.

[0212] The embodiments described in this application enable the loading volume to be obtained from the target point cloud, and then the loading rate to be determined from the loading volume. The loading rate obtained in this way is more accurate.

[0213] Figure 18 This is a schematic diagram of a load rate determination device based on a neural network provided in an embodiment of this application. The device is applied to an edge computing server and includes: an initial point cloud acquisition module 1801, an initial point cloud filtering module 1802, a vertex point cloud prediction module 1803, a vertex point cloud processing module 1804, a load rate determination module 1805, and a load information uploading module 1806.

[0214] The initial point cloud acquisition module 1801 is used to acquire an initial point cloud, wherein the initial point cloud includes a point cloud generated by an electromagnetic wave device located on the outer side of the rear of the carriage detecting the area within the field of view of the electromagnetic wave device.

[0215] The initial point cloud filtering module 1802 is used to filter the initial point cloud according to the truck parking area and the target height to obtain a preprocessed point cloud, wherein the target height is greater than or equal to the sum of the truck height and the first height of the electromagnetic wave device from the ground.

[0216] Vertex point cloud prediction module 1803 is used to predict the vertex point cloud of the carriage in the preprocessed point cloud using a preset neural network model, wherein the vertex point cloud includes the preprocessed point cloud corresponding to the preset vertices of the carriage.

[0217] Vertex point cloud processing module 1804 is used to process the preprocessed point cloud according to the vertex point cloud to obtain a target point cloud, wherein the target point cloud includes the point cloud of the truck car and the interior of the truck car.

[0218] The loading rate determination module 1805 is used to determine the cargo volume of the truck compartment using the target point cloud, and to determine the loading rate of the truck compartment based on the cargo volume.

[0219] The loading information upload module 1806 is used to upload vehicle loading information, including at least the stated loading rate, to the loading center platform.

[0220] Figure 19 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 19 The illustrated electronic device 1900 includes at least one processor 1901, a memory 1902, at least one network interface 1904, and a user interface 1903. The various components in the electronic device 1900 are coupled together via a bus system 1905. It is understood that the bus system 1905 is used to implement communication between these components. In addition to a data bus, the bus system 1905 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 19 The general labeled all buses as Bus System 1905.

[0221] The user interface 1903 may include a display, keyboard or clicking device (e.g., mouse, trackball), touchpad or touch screen, etc.

[0222] It is understood that the memory 1902 in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate Synchronous DRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 1902 described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0223] In some implementations, memory 1902 stores elements, executable units or data structures, or subsets thereof, or extended sets thereof: operating system 19021 and application program 19022.

[0224] The operating system 19021 includes various system programs, such as the framework layer, core library layer, and driver layer, used to implement various basic business functions and handle hardware-based tasks. The application program 19022 includes various applications, such as a media player and a browser, used to implement various application functions. Programs implementing the methods of the embodiments of this application can be included in application program 19022.

[0225] In this embodiment, by calling the program or instructions stored in memory 1902, specifically the program or instructions stored in application program 19022, processor 1901 executes the method steps provided in each method embodiment, including, for example:

[0226] Acquire an initial point cloud, wherein the initial point cloud includes a point cloud generated by an electromagnetic wave device located on the outer side of the rear of the carriage, which detects the area within the field of view of the electromagnetic wave device;

[0227] Based on the truck parking area and target height, the initial point cloud is filtered to obtain a preprocessed point cloud, wherein the target height is greater than or equal to the sum of the truck height and the first height of the electromagnetic wave device from the ground.

[0228] Using a preset neural network model, the vertex point cloud of the carriage in the preprocessed point cloud is predicted, wherein the vertex point cloud includes the preprocessed point cloud corresponding to the preset vertices of the carriage.

[0229] The preprocessed point cloud is processed based on the vertex point cloud to obtain the target point cloud, wherein the target point cloud includes the point cloud of the truck compartment and the interior of the truck compartment.

[0230] The cargo volume of the truck bed is determined using the target point cloud, and the loading rate of the truck bed is determined based on the cargo volume.

[0231] Upload vehicle loading information, including at least the aforementioned loading rate, to the loading center platform.

[0232] The methods disclosed in the embodiments of this application can be applied to or implemented by the processor 1901. The processor 1901 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in the processor 1901 or by instructions in the form of software. The processor 1901 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software units in the decoding processor. The software units may be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 1902. Processor 1901 reads the information in memory 1902 and, in conjunction with its hardware, completes the steps of the above method.

[0233] It is understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions of this application, or combinations thereof.

[0234] For software implementation, the techniques described herein can be implemented through units that perform the functions described herein. The software code can be stored in memory and executed by a processor. The memory can be implemented within the processor or external to the processor.

[0235] The electronic device provided in this embodiment may be as follows: Figure 19 The electronic device shown can perform the following: Figure 4 , 8 This involves all the steps of the neural network-based load rate determination method in sections 14 and 17, thereby achieving... Figure 4 , 8 For details on the technical effects of the neural network-based load rate determination methods in sections 14 and 17, please refer to [link / reference needed]. Figure 4 , 8 The descriptions related to points 14 and 17 are brief and will not be elaborated upon here.

[0236] This application also provides a storage medium (computer-readable storage medium). This storage medium stores one or more programs. The storage medium may include volatile memory, such as random access memory; it may also include non-volatile memory, such as read-only memory, flash memory, hard disk, or solid-state drive; and it may also include combinations of the above types of memory.

[0237] When one or more programs in the storage medium can be executed by one or more processors to implement the above-described neural network-based load rate determination method executed on the electronic device side.

[0238] The processor executes a neural network-based load rate determination program stored in memory to implement the following steps of a neural network-based load rate determination method executed on the electronic device side:

[0239] Acquire an initial point cloud, wherein the initial point cloud includes a point cloud generated by an electromagnetic wave device located on the outer side of the rear of the carriage, which detects the area within the field of view of the electromagnetic wave device;

[0240] Based on the truck parking area and target height, the initial point cloud is filtered to obtain a preprocessed point cloud, wherein the target height is greater than or equal to the sum of the truck height and the first height of the electromagnetic wave device from the ground.

[0241] Using a preset neural network model, the vertex point cloud of the carriage in the preprocessed point cloud is predicted, wherein the vertex point cloud includes the preprocessed point cloud corresponding to the preset vertices of the carriage.

[0242] The preprocessed point cloud is processed based on the vertex point cloud to obtain the target point cloud, wherein the target point cloud includes the point cloud of the truck compartment and the interior of the truck compartment.

[0243] The cargo volume of the truck bed is determined using the target point cloud, and the loading rate of the truck bed is determined based on the cargo volume.

[0244] Upload vehicle loading information, including at least the aforementioned loading rate, to the loading center platform.

[0245] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0246] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0247] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above description is only a specific embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for determining load rate based on neural networks, characterized in that, Applied to edge computing servers, the method includes: Acquire an initial point cloud, wherein the initial point cloud includes a point cloud generated by an electromagnetic wave device located on the outer side of the rear of the carriage, which detects the area within the field of view of the electromagnetic wave device; Based on the truck parking area and target height, the initial point cloud is filtered to obtain a preprocessed point cloud, wherein the target height is greater than or equal to the sum of the truck height and the first height of the electromagnetic wave device from the ground. Using a preset neural network model, the vertex point cloud of the carriage in the preprocessed point cloud is predicted, wherein the vertex point cloud includes the preprocessed point cloud corresponding to the preset vertices of the carriage. The preprocessed point cloud is processed based on the vertex point cloud to make the lower right corner vertex of the truck body coincide with the origin of the coordinate system, thereby obtaining the target point cloud. The target point cloud includes the point cloud of the truck body and the point cloud inside the truck body. Determining the cargo volume of a truck bed using the target point cloud and determining the loading rate of the truck bed based on the cargo volume includes: voxel downsampling the target point cloud to obtain a voxel downsampled point cloud; obtaining the actual width and actual height of the truck bed based on the voxel downsampled point cloud; constructing an M*N empty matrix, where M includes the actual width and N includes the actual height; assigning the target point cloud to the empty matrix according to a preset assignment rule; and summing the assigned empty matrix to obtain the loading volume of the truck bed. Upload vehicle loading information, including at least the aforementioned loading rate, to the loading center platform.

2. The method according to claim 1, characterized in that, The preset neural network model is obtained in the following way: Acquire an initial sample point cloud, wherein the initial sample point cloud includes a sample point cloud generated by an electromagnetic wave device probing a region within the field of view of the electromagnetic wave device; Based on the truck parking area and target height, the initial sample point cloud is filtered to obtain the preprocessed sample point cloud; The vertex sample point cloud of the carriage in the preprocessed sample point cloud is labeled, wherein the vertex sample point cloud includes the preprocessed sample point cloud corresponding to the preset vertices of the carriage; Based on the labeled preprocessed sample point cloud, the preset neural network initial model is trained to obtain the preset neural network model; The preset neural network initial model includes a point cloud classification network, the feature extraction network of the point cloud classification network is a backbone, the point cloud classification network includes multiple fully connected layers, and the output layer includes multiple neurons corresponding to vertex sample point clouds. The ReLU activation function is used between the output layer and the fully connected layer.

3. The method according to claim 1, characterized in that, The step of processing the preprocessed point cloud based on the vertex point cloud to obtain the target point cloud includes: Based on the vertex point cloud, a minimum bounding box is fitted, and the preprocessed point cloud located within the minimum bounding box is selected as the target point cloud from the preprocessed point cloud.

4. The method according to claim 1, characterized in that, The step of processing the preprocessed point cloud based on the vertex point cloud to obtain the target point cloud includes: Based on the vertex point cloud, determine the angle and first distance corresponding to the preprocessed point cloud; The preprocessed point cloud is rotated by the angle around the second coordinate axis toward the first coordinate axis; The preprocessed point cloud, after being rotated by the aforementioned angle, is translated along the negative direction of the third coordinate axis by the first distance. Determine the second distance corresponding to the preprocessed point cloud after translating the first distance; The preprocessed point cloud, after being translated by the first distance, is translated by the second distance along the negative direction of the first coordinate axis. Determine the third distance corresponding to the preprocessed point cloud after translating the second distance; The preprocessed point cloud, after being translated by the second distance, is translated by the third distance along the positive direction of the second coordinate axis to obtain the target point cloud.

5. The method according to claim 4, characterized in that, The step of determining the angle and first distance corresponding to the preprocessed point cloud based on the vertex point cloud includes: By performing a planar projection on the vertex point cloud, a top view of the truck bed is obtained; By performing line fitting on the top view, the first line equations corresponding to the first length and width boundaries of the truck body are obtained; Based on the equation of the first straight line corresponding to the first long boundary on the right side of the truck body, the angle between the first long boundary on the right side and the first coordinate axis is determined, and the angle is determined to be the angle corresponding to the preprocessed point cloud. Rotate the top view by the angle along the first coordinate axis to obtain the rotated top view; By performing line fitting on the rotated top view, the second line equations corresponding to the second length and width boundaries of the truck body are obtained; Based on the equation of the second straight line corresponding to the second long boundary on the right side of the truck body, the first distance between the second long boundary on the right side and the first coordinate axis is determined, and the first distance is determined to be the first distance corresponding to the preprocessed point cloud.

6. The method according to claim 5, characterized in that, The determination of the second distance corresponding to the preprocessed point cloud after translating the first distance includes: Based on the second straight line equation corresponding to the second wide boundary of the rear side of the truck body, determine the second distance between the second wide boundary of the rear side and the electromagnetic wave device; The second distance is determined to be the second distance corresponding to the preprocessed point cloud after translating the first distance.

7. The method according to claim 4, characterized in that, The third distance corresponding to the preprocessed point cloud after determining the translation of the second distance includes: The preprocessed point cloud, after being translated by the second distance, is then projected onto a plane to obtain a front view of the truck bed. By performing a straight line fit on the front view, the equations of the third straight line corresponding to the height and width boundaries of the truck body are obtained; Based on the equation of the third straight line corresponding to the lower width boundary of the truck body, determine the second height between the lower width boundary and the third coordinate axis; The difference between the first height and the second height of the electromagnetic wave device above the ground is obtained to obtain the third height, and the third height is determined to be the third distance corresponding to the preprocessed point cloud after translating the second distance.

8. The method according to claim 1, characterized in that, The step of assigning the target point cloud to the empty matrix according to a preset assignment rule includes: Using the plane corresponding to the actual width and the actual height as a reference plane, for all target point clouds corresponding to any smallest unit in the reference plane, the target point cloud closest to the reference plane is selected and recorded in the corresponding position of the empty matrix corresponding to the smallest unit.

9. A device for determining loading rate based on neural networks, characterized in that, The device, applied to an edge computing server, includes: An initial point cloud acquisition module is used to acquire an initial point cloud, wherein the initial point cloud includes a point cloud generated by an electromagnetic wave device located on the outer side of the rear of the carriage detecting the area within the field of view of the electromagnetic wave device; The initial point cloud filtering module is used to filter the initial point cloud according to the truck parking area and the target height to obtain a preprocessed point cloud, wherein the target height is greater than or equal to the sum of the truck height and the first height of the electromagnetic wave device from the ground. The vertex point cloud prediction module is used to predict the vertex point cloud of the carriage in the preprocessed point cloud using a preset neural network model, wherein the vertex point cloud includes the preprocessed point cloud corresponding to the preset vertices of the carriage. The vertex point cloud processing module is used to process the preprocessed point cloud according to the vertex point cloud, and to process the preprocessed point cloud according to the vertex point cloud so that the lower right corner vertex of the car coincides with the origin of the coordinate system to obtain the target point cloud, wherein the target point cloud includes the point cloud of the freight car and the interior of the freight car. The loading rate determination module is used to determine the cargo volume of the truck compartment using the target point cloud, and to determine the loading rate of the truck compartment based on the cargo volume. Specifically, it is used to: perform voxel downsampling on the target point cloud to obtain a voxel downsampled point cloud; obtain the actual width and actual height of the truck compartment based on the voxel downsampled point cloud; construct an M*N empty matrix, where M includes the actual width and N includes the actual height; assign the target point cloud to the empty matrix according to a preset assignment rule; and sum the assigned empty matrix to obtain the loading volume of the truck compartment. The loading information upload module is used to upload vehicle loading information, including at least the loading rate, to the loading center platform.

Images