Vehicle loading rate identification method, device, electronic device and storage medium

Abstract

The present application provides a vehicle loading ratio identification method, apparatus, electronic device, and computer-readable storage medium. The vehicle loading ratio identification method includes: acquiring a first image of a vehicle compartment area containing a vehicle to be identified, and acquiring a second image containing the exterior appearance of the vehicle to be identified; performing feature extraction processing based on the first and second images to obtain target feature information of the vehicle to be identified; and performing loading ratio prediction processing based on the target feature information to obtain a first loading ratio of the vehicle to be identified. This application can improve the accuracy of vehicle loading ratio identification.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a vehicle loading rate identification method, device, electronic device and computer-readable storage medium. Background Art

[0002] In logistics transportation, the efficiency of logistics transportation greatly affects the timeliness of goods.

[0003] Loading factor is a method for evaluating operational efficiency. In trucking, it generally refers to the actual volume of cargo carried divided by the vehicle's maximum cargo capacity. In logistics, a vehicle's loading factor directly impacts overall transport efficiency, making accurate loading factor crucial for efficient logistics.

[0004] Currently, existing methods for detecting vehicle loading rates using vehicle compartment images have emerged. However, the inventors of this application have discovered in actual applications that, due to the varying loading capacities of different vehicles, existing methods for detecting vehicle loading rates based on vehicle compartment images rely on a single set of image feature information, resulting in low detection accuracy for vehicle loading rates. Summary of the Invention

[0005] The present application provides a vehicle loading rate identification method, device, electronic device and computer-readable storage medium, aiming to solve the problem of low vehicle loading rate detection accuracy due to the single image feature information based on the existing vehicle loading rate detection method.

[0006] In a first aspect, the present application provides a vehicle loading rate identification method, the method comprising:

[0007] Acquire a first image containing a vehicle compartment area of a vehicle to be identified, and acquire a second image containing an appearance of the vehicle to be identified;

[0008] Performing feature extraction processing based on the first image and the second image to obtain target feature information of the vehicle to be identified;

[0009] A loading rate prediction process is performed based on the target feature information to obtain a first loading rate of the vehicle to be identified.

[0010] In a possible implementation of the present application, performing feature extraction processing based on the first image and the second image to obtain target feature information of the vehicle to be identified includes:

[0011] calling a first feature extraction layer in the trained loading rate prediction network to perform feature extraction processing on the first image to obtain first feature information of the first image;

[0012] calling a second feature extraction layer in the trained loading rate prediction network to perform feature extraction processing on the second image to obtain second feature information of the second image;

[0013] The feature fusion layer in the trained loading rate prediction network is called to perform fusion processing according to the first feature information and the second feature information to obtain the target feature information.

[0014] In a possible implementation of the present application, a third image is further included, wherein the calling of the feature fusion layer in the trained loading rate prediction network performs fusion processing based on the first feature information and the second feature information to obtain the target feature information includes:

[0015] acquiring a third image containing the cargo loaded in the vehicle to be identified;

[0016] calling a third feature extraction layer in the trained loading rate prediction network to perform feature extraction processing on the third image to obtain third feature information of the third image;

[0017] The feature fusion layer in the trained loading rate prediction network is called to perform fusion processing according to the first feature information, the second feature information, and the third feature information to obtain the target feature information.

[0018] In a possible implementation of the present application, performing loading rate prediction processing according to the target feature information to obtain the first loading rate of the vehicle to be identified includes:

[0019] Calling the prediction layer in the trained load rate prediction network to perform prediction processing based on the target feature information to obtain a first confidence level of a preset load rate category;

[0020] Obtaining a target loading rate category having the maximum first confidence level from the loading rate categories;

[0021] A first loading rate of the vehicle to be identified is determined according to the target loading rate category.

[0022] In a possible implementation of the present application, determining the first loading rate of the vehicle to be identified according to the target loading rate category includes:

[0023] According to the target loading rate category and the preset loading rate interval, the first loading rate of the vehicle to be identified is determined, wherein the loading rate category has N and the loading rate interval is The loading rate category is represented by a numerical value M;

[0024] The determining, according to the target loading rate category and a preset loading rate interval, a first loading rate of the vehicle to be identified includes:

[0025] The first loading rate is determined according to the following formula:

[0026]

[0027] Among them, Y represents the first loading rate, N represents the number of the loading rate categories, and M0 represents the representative value of the target loading rate category.

[0028] In a possible implementation of the present application, the method further includes:

[0029] Acquiring a sample image, wherein the sample image includes a vehicle compartment area image of the sample vehicle and an appearance image of the sample vehicle;

[0030] calling a first feature extraction layer in a preset loading rate prediction network to perform feature extraction processing on the carriage area image to obtain fourth feature information of the carriage area image;

[0031] calling a second feature extraction layer in a preset loading rate prediction network to perform feature extraction processing on the appearance image to obtain fifth feature information of the appearance image;

[0032] calling a feature fusion layer in a preset loading rate prediction network to perform fusion processing based on the fourth feature information and the fifth feature information to obtain image feature information of the sample image;

[0033] Calling a prediction layer in a preset load rate prediction network to perform prediction processing based on the image feature information to obtain vehicle type information corresponding to the sample image and a second load rate;

[0034] The preset loading rate prediction network is trained according to the vehicle type information and the second loading rate to obtain the trained loading rate prediction network.

[0035] In a possible implementation of the present application, performing prediction processing based on the image feature information to obtain the vehicle type information and the second loading rate corresponding to the image to be trained includes:

[0036] Performing vehicle type prediction processing based on the image feature information to obtain a second confidence level of a preset vehicle type category; performing loading rate prediction processing based on the image feature information to obtain a third confidence level of a preset loading rate category;

[0037] The vehicle type category with the second highest confidence is obtained as the vehicle type information; and the loading rate category with the third highest confidence is obtained as the second loading rate.

[0038] In a second aspect, the present application provides a vehicle loading rate identification device, the vehicle loading rate identification device comprising:

[0039] an acquisition unit, configured to acquire a first image of a vehicle compartment area including a vehicle to be identified, and acquire a second image including an appearance of the vehicle to be identified;

[0040] a feature extraction unit, configured to perform feature extraction processing based on the first image and the second image to obtain target feature information of the vehicle to be identified;

[0041] The prediction unit is used to perform loading rate prediction processing according to the target feature information to obtain a first loading rate of the vehicle to be identified.

[0042] In a possible implementation of the present application, the feature extraction unit is further configured to:

[0043] calling a first feature extraction layer in the trained loading rate prediction network to perform feature extraction processing on the first image to obtain first feature information of the first image;

[0044] calling a second feature extraction layer in the trained loading rate prediction network to perform feature extraction processing on the second image to obtain second feature information of the second image;

[0045] The feature fusion layer in the trained loading rate prediction network is called to perform fusion processing according to the first feature information and the second feature information to obtain the target feature information.

[0046] In a possible implementation of the present application, a third image is further included, and the feature extraction unit is further configured to:

[0047] acquiring a third image containing the cargo loaded in the vehicle to be identified;

[0048] calling a third feature extraction layer in the trained loading rate prediction network to perform feature extraction processing on the third image to obtain third feature information of the third image;

[0049] The feature fusion layer in the trained loading rate prediction network is called to perform fusion processing according to the first feature information, the second feature information, and the third feature information to obtain the target feature information.

[0050] In a possible implementation of the present application, the prediction unit is further configured to:

[0051] Calling the prediction layer in the trained load rate prediction network to perform prediction processing based on the target feature information to obtain a first confidence level of a preset load rate category;

[0052] Obtaining a target loading rate category having the maximum first confidence level from the loading rate categories;

[0053] A first loading rate of the vehicle to be identified is determined according to the target loading rate category.

[0054] In a possible implementation of the present application, the prediction unit is further configured to:

[0055] According to the target loading rate category and the preset loading rate interval, the first loading rate of the vehicle to be identified is determined, wherein the loading rate category has N and the loading rate interval is The loading rate category is represented by a numerical value M;

[0056] The determining, according to the target loading rate category and a preset loading rate interval, a first loading rate of the vehicle to be identified includes:

[0057] The first loading rate is determined according to the following formula:

[0058]

[0059] Among them, Y represents the first loading rate, N represents the number of the loading rate categories, and M0 represents the representative value of the target loading rate category.

[0060] In a possible implementation of the present application, the vehicle loading rate identification device further includes a training unit, wherein the training unit is specifically configured to:

[0061] Acquiring a sample image, wherein the sample image includes a vehicle compartment area image of the sample vehicle and an appearance image of the sample vehicle;

[0062] calling a first feature extraction layer in a preset loading rate prediction network to perform feature extraction processing on the carriage area image to obtain fourth feature information of the carriage area image;

[0063] calling a second feature extraction layer in a preset loading rate prediction network to perform feature extraction processing on the appearance image to obtain fifth feature information of the appearance image;

[0064] calling a feature fusion layer in a preset loading rate prediction network to perform fusion processing based on the fourth feature information and the fifth feature information to obtain image feature information of the sample image;

[0065] Calling a prediction layer in a preset load rate prediction network to perform prediction processing based on the image feature information to obtain vehicle type information corresponding to the sample image and a second load rate;

[0066] The preset loading rate prediction network is trained according to the vehicle type information and the second loading rate to obtain the trained loading rate prediction network.

[0067] In a possible implementation of the present application, the training unit is further configured to:

[0068] Performing vehicle type prediction processing based on the image feature information to obtain a second confidence level of a preset vehicle type category; performing loading rate prediction processing based on the image feature information to obtain a third confidence level of a preset loading rate category;

[0069] The vehicle type category with the second highest confidence is obtained as the vehicle type information; and the loading rate category with the third highest confidence is obtained as the second loading rate.

[0070] In a third aspect, the present application also provides an electronic device comprising a processor and a memory, wherein the memory stores a computer program, and when the processor calls the computer program in the memory, it executes the steps of any vehicle loading rate identification method provided in the present application.

[0071] In a fourth aspect, the present application also provides a computer-readable storage medium having a computer program stored thereon, and the computer program is loaded by a processor to execute the steps in the vehicle loading rate identification method.

[0072] The present application obtains a first image containing the cabin area of the vehicle to be identified, and obtains a second image containing the appearance of the vehicle to be identified; performs feature extraction processing based on the first image and the second image to obtain target feature information of the vehicle to be identified; performs loading rate prediction processing based on the target feature information to obtain a first loading rate of the vehicle to be identified. On the one hand, there are large differences in the loading capacity of vehicles of different models. Since the loading rate information of different models can be identified through the appearance image of the vehicle (i.e., the second image), the model as auxiliary information can help distinguish the loading rate differences of different models. By combining the features of the cabin area and the appearance of the vehicle to be identified to predict the loading rate, the prediction accuracy of the vehicle loading rate is improved. On the other hand, due to the overall combination of the first image and the second image, the feature expression capability of the loading rate recognition scene is enhanced, thereby improving the recognition accuracy of the vehicle loading rate to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0074] Figure 1 This is a schematic diagram of a scenario of a vehicle loading rate identification system provided in an embodiment of the present application;

[0075] Figure 2 This is a flow chart of a vehicle loading rate identification method provided in an embodiment of the present application;

[0076] Figure 3 This is a schematic diagram of a scene of the vehicle compartment area provided in an embodiment of the present application;

[0077] Figure 4 This is a schematic diagram of a scene of the appearance of a vehicle provided in an embodiment of the present application;

[0078] Figure 5 This is a flowchart of an embodiment of step S20 provided in the embodiments of the present application;

[0079] Figure 6 1 is a schematic diagram comparing the single-image predicted loading rate and the multi-image predicted loading rate provided by the embodiment of the present application;

[0080] Figure 7 This is a schematic structural diagram of an embodiment of a vehicle loading rate identification device provided in an embodiment of the present application;

[0081] Figure 8 It is a schematic diagram of the structure of an embodiment of an electronic device provided in the embodiments of the present application. DETAILED DESCRIPTION

[0082] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0083] In the description of the embodiments of the present application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0084] In order to enable any person skilled in the art to implement and use the present application, the following description is provided. In the following description, details are listed for the purpose of explanation. It should be understood that one of ordinary skill in the art will recognize that the present application can be implemented without using these specific details. In other examples, well-known processes will not be elaborated in detail to avoid obscuring the description of the embodiments of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest range of principles and features disclosed in accordance with the embodiments of the present application.

[0085] The embodiments of the present application provide a vehicle loading rate identification method, device, electronic device, and computer-readable storage medium. The vehicle loading rate identification device can be integrated into an electronic device, which can be a server, a terminal, or other device.

[0086] First of all, before introducing the embodiments of the present application, the relevant content about the application background of the embodiments of the present application is introduced.

[0087] The vehicle loading rate can be divided into instantaneous loading rate and process loading rate. The instantaneous loading rate refers to the vehicle loading rate value given when the vehicle arrives at or leaves the port, while the process loading rate refers to the real-time loading rate of the vehicle during the cargo loading and unloading process.

[0088] In the logistics field, loading rates can reflect the working conditions of logistics transfer stations. Specifically, the instantaneous loading rate of carriages can be used to monitor the loading rate of carriages at the time of departure to rationally dispatch vehicles, reduce logistics costs, fully utilize vehicle resources, and improve operational efficiency. The process loading rate can also monitor the current loading and unloading status of vehicles in real time, analyze loading and unloading efficiency, optimize labor scheduling, and save labor costs.

[0089] Currently, in the logistics field, there are three main methods for estimating loading rates: the first is to estimate the loading rate of the vehicle compartment by calculating the ratio of the weight of the cargo loaded in the truck to the rated load of the truck; the second is to rely on manual evaluation; and the third is to rely on auxiliary measuring equipment for measurement. The first method is prone to large errors in situations where there is no relatively fixed ratio between the volume and weight of the cargo, such as large volume and small weight or small volume and large weight, and requires additional weighing equipment. The second method relies on the workers' experience for estimation, which brings additional labor costs, and the estimation results are easily affected by the workers' subjective consciousness. The third method measures the volume of the cargo in the compartment by building a dedicated laser measurement platform. This method requires a large cost, has certain requirements for the installation site, and has a slow scanning and measurement speed.

[0090] Based on the above-mentioned defects of the existing related technologies, the embodiments of the present application provide a vehicle loading rate identification method. The executor of the vehicle loading rate identification method of the embodiments of the present application can be the vehicle loading rate identification device provided in the embodiments of the present application, or a server device, a physical host or a user equipment (UE) and other different types of electronic devices that integrate the vehicle loading rate identification device. Among them, the vehicle loading rate identification device can be implemented in hardware or software, and the UE can specifically be a terminal device such as a smart phone, a tablet computer, a laptop computer, a PDA, a desktop computer or a personal digital assistant (PDA).

[0091] The electronic device can operate independently or in a cluster. By applying the vehicle loading rate identification method provided in the embodiments of this application, the accuracy of vehicle loading rate identification can be improved. Specifically, when applied to the logistics field, it can improve the rationality of vehicle scheduling, reduce logistics costs, fully utilize vehicle resources, and improve operational efficiency.

[0092] See also Figure 1 , Figure 1 Schematic diagram of a vehicle loading ratio recognition system provided in an embodiment of the present application. The vehicle loading ratio recognition system may include an electronic device 100, which incorporates a vehicle loading ratio recognition device. For example, the electronic device may capture a first image of a vehicle compartment area containing a vehicle to be recognized, and a second image containing the exterior appearance of the vehicle to be recognized; perform feature extraction processing based on the first and second images to obtain target feature information of the vehicle to be recognized; and perform prediction processing based on the target feature information to obtain a first loading ratio of the vehicle to be recognized.

[0093] In addition, if Figure 1 As shown, the vehicle loading rate identification system may further include a memory 200 for storing data, such as image data and video data.

[0094] It should be noted that Figure 1 The scenario diagram of the vehicle loading rate identification system shown is only an example. The vehicle loading rate identification system and scenario described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided by the embodiment of the present application. Ordinary technicians in this field can know that with the evolution of the vehicle loading rate identification system and the emergence of new business scenarios, the technical solution provided by the embodiment of the present application is also applicable to similar technical problems.

[0095] Next, we will begin to introduce the vehicle loading rate identification method provided in the embodiment of the present application. In the embodiment of the present application, an electronic device is used as the execution subject. For the sake of simplicity and ease of description, this execution subject will be omitted in the subsequent method embodiments.

[0096] Reference Figure 2 , Figure 2 This is a flow chart of a vehicle loading rate identification method provided in an embodiment of the present application. It should be noted that although a logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order than shown here. The vehicle loading rate identification method includes steps S10 to S30, wherein:

[0097] S10: Acquire a first image of a vehicle compartment area including a vehicle to be identified, and acquire a second image including an appearance of the vehicle to be identified.

[0098] The first image refers to the image of the vehicle compartment area to be identified. Figure 3 , Figure 3 This is a schematic diagram of a scene of the vehicle compartment area provided in an embodiment of the present application. Figure 3 The middle dashed box represents the vehicle compartment area.

[0099] The second image is an image of the appearance of the vehicle to be identified. Figure 4 , Figure 4 This is a scene schematic diagram of the appearance of a vehicle provided in an embodiment of the present application. Figure 4 The figure shows the appearance of three different vehicle models.

[0100] Specifically, in actual applications, the electronic device that applies the vehicle loading rate identification method provided in the embodiment of the present application may directly include a camera in hardware (the camera can be used to collect images of the vehicle's compartment area, appearance images, images of the cargo loaded inside the vehicle, etc.), and store the images taken by the camera locally, which can be directly read inside the electronic device; alternatively, the electronic device may also establish a network connection with the camera and obtain the images obtained by the camera online from the camera based on the network connection; alternatively, the electronic device may also read the images obtained by the camera from a related storage medium that stores the images obtained by the camera. The specific acquisition method is not limited here.

[0101] The camera can capture images according to a preset shooting method, such as setting the shooting height, shooting direction, or shooting distance. The specific shooting method can be adjusted according to the camera itself and is not limited here. The multiple frames of images captured by the camera can be combined into a video through a timeline.

[0102] S20: Perform feature extraction processing based on the first image and the second image to obtain target feature information of the vehicle to be identified.

[0103] The target feature information refers to the image features of the vehicle to be identified obtained after feature extraction is performed based on the first image and the second image.

[0104] Specifically, in some embodiments, a loading rate prediction network that has undergone deep learning can be used to predict the first loading rate of the vehicle to be identified; wherein, the loading rate prediction network (if not otherwise specified, the loading rate prediction network referred to in this article refers to the trained loading rate prediction network.) can be trained by following the steps A1 to A6.

[0105] The load rate prediction network may include a feature extraction layer, a feature fusion layer, and a prediction layer. The feature extraction layer may further include a first feature extraction layer and a second feature extraction layer.

[0106] The first feature extraction layer is used to output the first feature information of the first image (i.e., the image of the compartment area) based on the first image (i.e., the image of the compartment area). The first feature extraction layer takes the first image (i.e., the image of the compartment area) as input, and performs one or more operations including but not limited to convolution, pooling, etc. on the first image to achieve feature extraction of the first image to obtain the first feature information of the first image (i.e., the image feature of the compartment area). The first feature extraction layer can be a convolutional neural network (CNN). For example, the CNN network (denoted as CNN1) obtained by directly using the classification network ResNet50 with the fully connected layer (denoted as FC) removed can be used as the first feature extraction layer.

[0107] The second feature extraction layer is used to output second feature information of the second image (i.e., image features of the vehicle's appearance) based on the second image (i.e., image of the vehicle's appearance). The second feature extraction layer takes the second image (i.e., image of the vehicle's appearance) as input and performs one or more operations including but not limited to convolution and pooling on the second image to extract features from the second image to obtain the second feature information of the second image (i.e., image features of the vehicle's appearance). The second feature extraction layer can also be a convolutional neural network. For example, the CNN network (denoted as CNN2) obtained by directly removing the fully connected layer (denoted as FC) from the classification network ResNet50 can be used as the second feature extraction layer.

[0108] The feature fusion layer is used to output the target feature information of the vehicle to be identified (i.e., the image features of the vehicle) based on the first feature information and the second feature information. The feature fusion layer takes the first feature information and the second feature information (i.e., the image features of the vehicle appearance and the image features of the vehicle cabin area) as input and fuses the first feature information and the second feature information in series to obtain the target feature information of the vehicle to be identified (i.e., the image features of the vehicle).

[0109] The prediction layer is used to output the first load factor of the vehicle to be identified based on the target feature information. The prediction layer takes the target feature information as input and performs regression classification prediction processing based on the target feature information to determine the first load factor of the vehicle to be identified.

[0110] In some embodiments, the load rate prediction network may include a feature extraction layer and a prediction layer. The feature extraction layer may further include a first feature extraction layer and a second feature extraction layer. The functional implementation of the feature extraction layer and the prediction layer can refer to the above description and will not be repeated here. The difference is that in this case, there is no need to fuse the first feature information and the second feature information. The target feature information directly includes the first feature information output by the first feature extraction layer and the second feature information output by the second feature extraction layer. The prediction layer directly performs regression classification prediction processing based on the first feature information and the second feature information to determine the first load rate of the vehicle to be identified.

[0111] Please refer to Figure 5 , Figure 5 This is a flow chart of an embodiment of step S20 provided in the embodiment of the present application. In this case, step S20 can be implemented by the feature extraction layer in the load rate prediction network, that is, step S20 can specifically include the following steps S21 to S23, wherein:

[0112] S21. Calling a first feature extraction layer in the trained loading rate prediction network to perform feature extraction processing on the first image to obtain first feature information of the first image.

[0113] The first feature information refers to image features obtained after performing feature extraction processing on the image of the vehicle compartment area to be identified (ie, the first image), such as convolution, pooling and other operations.

[0114] Please refer to Figure 6 , Figure 6 This is a schematic diagram comparing the single image prediction loading rate and the multi-image prediction loading rate provided by the embodiment of the present application. Figure 6 (a) shows the prediction of the vehicle loading rate based on the vehicle cabin area image. Figure 6 (b) shows the prediction of the vehicle loading rate based on the vehicle cabin area image and the appearance image.

[0115] Depend on Figure 6 (a) It can be seen that if the vehicle loading rate is predicted based only on the vehicle compartment area image, it is only necessary to perform feature extraction on the vehicle compartment area image to obtain the image features of the vehicle compartment area image; then, based on the image features of the vehicle compartment area image, regression prediction processing can be directly performed to output the vehicle loading rate.

[0116] from Figure 6 As can be seen in (b), when predicting the vehicle's loading rate based on the vehicle's cabin area image and appearance image, in order to ensure that the loading rate prediction can be performed by combining the features of the cabin area and the vehicle's appearance, in addition to performing feature extraction processing on the cabin area image and the appearance image respectively, it is also necessary to further fuse the extracted features to obtain fused image features; then, regression prediction processing is performed based on the fused image features, and finally the vehicle's loading rate is output. The purpose of steps S21 to S22 in the embodiment of the present application is precisely to extract and fuse the features of the vehicle's cabin area and appearance, and the purpose of step S23 is to combine the fused features of the vehicle's cabin area and appearance to predict the vehicle's loading rate.

[0117] For example, the image of the compartment area of truck A (i.e., the first image) is input into the trained loading rate prediction network to call the first feature extraction layer in the loading rate prediction network, and one or more operations such as convolution and pooling are performed on the image of the compartment area of truck A to achieve feature extraction of the image of the compartment area of truck A to obtain the image features of the compartment area of truck A (i.e., the first feature information).

[0118] S22: Calling a second feature extraction layer in the trained loading rate prediction network to perform feature extraction processing on the second image to obtain second feature information of the second image.

[0119] The second feature information refers to image features obtained after performing feature extraction processing on the image of the appearance of the vehicle to be identified (ie, the second image), such as convolution, pooling and other operations.

[0120] For example, the image of the appearance of truck A (i.e., the second image) is input into the trained loading rate prediction network to call the second feature extraction layer in the loading rate prediction network, and one or more operations such as convolution and pooling are performed on the image of the appearance of truck A to achieve feature extraction of the image of the appearance of truck A to obtain the image features of the appearance of truck A (i.e., the second feature information).

[0121] S23: Calling a feature fusion layer in the trained loading rate prediction network to perform fusion processing based on the first feature information and the second feature information to obtain the target feature information.

[0122] For example, first, an RGB image of size 224*224 is input to the feature extraction layer (specifically, the first image is input to the first feature extraction layer such as CNN1, and the second image is input to the second feature extraction layer such as CNN2) for convolution calculation. After 4 layers of convolution processing, a feature of size [1,2048,7,7] is obtained, and then after pooling processing, a feature of size [1,2048] is obtained. At this time, the first feature information and second feature information corresponding to the first image and the second image are recorded as feature1 and feature2, respectively.

[0123] Then, feature1 and feature2 are input into the feature fusion layer in the trained loading rate prediction network to call the feature fusion layer to fuse feature1 and feature2 in series into a feature of size [1,4096], thereby obtaining the target feature information.

[0124] Finally, the target feature information is fed into the prediction layer to output the final result. The processing after the target feature information is fed into the prediction layer will be described in detail later and will not be repeated here.

[0125] Step S21 and step S22 may be performed simultaneously, or step S21 may be performed first and then step S22, or step S22 may be performed first and then step S21. To improve vehicle loading rate recognition, it may be considered to perform step S21 and step S22 simultaneously.

[0126] From the above content, it can be seen that after extracting features from the first image and the second image respectively to obtain the first feature information of the first image and the second feature information of the second image, the first feature information and the second feature information are fused to obtain the target feature information of the vehicle to be identified; on the one hand, when subsequently performing regression prediction of the vehicle loading rate based on the target feature information, it is possible to simultaneously combine the features of the cabin area and the appearance of the vehicle to be identified. On the other hand, since the appearance image of the vehicle allows the loading rate prediction network to identify the loading rate information of different models, the model as auxiliary information can help distinguish the loading rate differences of different models, thereby improving the prediction accuracy of the vehicle loading rate. On the other hand, through the overall combination of the first image and the second image, the feature expression ability of the loading rate recognition scene is enhanced, thereby improving the recognition accuracy of the vehicle loading rate to a certain extent.

[0127] S30: Perform loading rate prediction processing according to the target feature information to obtain a first loading rate of the vehicle to be identified.

[0128] The first loading rate refers to the loading rate of the vehicle to be identified obtained after loading rate prediction processing is performed based on target feature information.

[0129] Correspondingly, step S30 may be implemented by a prediction layer in the load rate prediction network. That is, step S30 may include the following steps S31 to S33, wherein:

[0130] S31, calling the prediction layer in the trained load rate prediction network, performing prediction processing based on the target feature information, and obtaining a first confidence level of a preset load rate category;

[0131] The first confidence level refers to the confidence level of a preset loading rate category obtained after prediction processing is performed based on target feature information.

[0132] In some embodiments, the number of output nodes of the prediction layer in the loading rate prediction network is set to N, wherein N1 output nodes correspond to outputting the confidence of preset vehicle type categories (such as small, medium and large vehicle types), and N2 output nodes correspond to outputting the confidence of preset loading rate categories (such as 0%, 5%, 10%,..., 100% with an interval of 5%, a total of 21 loading rates), N, N1, and N2 are all positive integers, and N=N1+N2.

[0133] The prediction layer performs prediction processing based on the target feature information, determines the confidence level of each preset vehicle type category and the first confidence level of each preset loading rate category, and outputs them to the corresponding output nodes.

[0134] For example, the first three output nodes of the prediction layer represent the vehicle type categories: small, medium, and large. The next 21 nodes represent the load factor categories: 0%, 5%, 10%, ..., 100%, for a total of 21 load factors. The first three nodes output the confidence level for the vehicle type being small, medium, and large: 0.5, 0.4, and 0.9, respectively. The fourth to 23rd nodes output the first confidence level for the load factor of 0%, 5%, 10%, ..., 100%, respectively: 0.2, 0.4, 0.9, ..., 0.1, respectively.

[0135] S32: Obtain a target loading rate category with the maximum first confidence from the loading rate categories.

[0136] For ease of understanding, the following example of step S31 is used. For example, among the first confidence levels output by the prediction layer: 0.2, 0.4, 0.9, ..., 0.1, the maximum value of the first confidence level is 0.9. The preset loading rate corresponding to the first confidence level of 0.9 is 10%, so 10% is selected as the target loading rate category.

[0137] S33. Determine a first loading rate of the vehicle to be identified according to the target loading rate category.

[0138] In some embodiments, the target loading rate category is represented by the corresponding vehicle loading rate (as exemplified in steps S31 and S32 above). In this case, the target loading rate category can be directly used as the first loading rate of the vehicle to be identified. For example, in the example of step S32, if the target loading rate category is 10%, 20% can be directly used as the first loading rate of the vehicle to be identified.

[0139] As can be seen from the above, because the load rate prediction network is trained through deep learning, it can fully learn the characteristic relationship between the image features of the vehicle's cabin area and exterior images and the first load rate of the vehicle to be identified. The prediction layer in the trained load rate prediction network performs prediction processing based on the target feature information to obtain the first confidence level of the preset load rate category. Then, from the preset load rate categories, the target load rate category with the highest first confidence level is obtained to determine the first load rate of the vehicle to be identified, ensuring the accuracy of the prediction of the first load rate.

[0140] In some embodiments, the target loading rate category is represented by a numerical value M (the numerical value M does not directly correspond to the vehicle loading rate). In this case, step S33 may specifically include: determining a first loading rate of the vehicle to be identified based on the target loading rate category and a preset loading rate interval.

[0141] There are N preset loading rate categories, and the preset loading rate interval is Each preset loading rate category is represented by a numerical value M;

[0142] Specifically, the first loading rate can be determined according to the following formula:

[0143]

[0144] Among them, Y represents the first loading rate, N represents the number of the loading rate categories, and M0 represents the representative value of the target loading rate category.

[0145] For example, the prediction layer in the load rate prediction network has a total of 21 final output nodes (i.e., the number of preset load rate categories N=21), and each output node outputs the first confidence level of a preset load rate category, that is, each output node outputs a preset load rate confidence level (e.g., the load rate is divided into: 0%, 5%, 10%, ..., 100% according to a granularity of 5%, and recorded in order as: 0, 1, 2, 3, ..., 19, 20, that is, the preset load rate categories are: 0, 1, 2, 3, ..., 19, 20). Among them, the category with the largest first confidence level is the target load rate category, and multiplying it by 5% is the first load rate of the vehicle to be identified. For example, if the load rate category with the largest first confidence level is "3", then multiplying it by the load rate division granularity of 5% (i.e., the preset load rate spacing) can determine the first load rate of the vehicle to be identified as: 3×5%=15%.

[0146] From the above, we can see that by setting the preset loading rate categories to N, the preset loading rate interval is Each preset loading rate category is represented by a numerical value M. After determining the target loading rate category with the largest first confidence level, the simple formula The calculation quickly determines the first loading rate of the vehicle to be identified. Furthermore, since there is no need to input complex loading rate values as loading rate categories for identifying loading rates during model training, the complexity of manual operations is reduced.

[0147] From the above content, it can be seen that by obtaining a first image containing the cabin area of the vehicle to be identified, and obtaining a second image containing the appearance of the vehicle to be identified; performing feature extraction processing based on the first image and the second image, the target feature information of the vehicle to be identified is obtained; and performing loading rate prediction processing based on the target feature information, the first loading rate of the vehicle to be identified is obtained. On the one hand, there are large differences in the loading capacity of vehicles of different models. Since the loading rate information of different models can be identified through the appearance image of the vehicle (i.e., the second image), the model as auxiliary information can help distinguish the loading rate differences of different models. By combining the features of the cabin area and appearance of the vehicle to be identified to predict the loading rate, the prediction accuracy of the vehicle loading rate is improved. On the other hand, due to the overall combination of the first image and the second image, the feature expression ability of the loading rate recognition scene is enhanced, thereby improving the recognition accuracy of the vehicle loading rate to a certain extent.

[0148] In addition, since the vehicle's appearance image can be used to allow the loading rate prediction network to identify the loading rate information of different vehicle models, the vehicle model can serve as auxiliary information to help distinguish the differences in loading rates under different vehicle models. The loading rate prediction is performed by combining the target features with the cabin area and appearance features of the vehicle to be identified, thereby improving the prediction accuracy of the vehicle's loading rate.

[0149] Taking into account the differences in volume of different cargoes, in order to further improve the recognition accuracy of vehicle loading rates, in some embodiments of the present application, a first image of the vehicle compartment area to be identified, a second image of the vehicle exterior to be identified, and a third image of the cargo loaded in the vehicle to be identified are combined to predict the loading rate of the vehicle to be identified.

[0150] Specifically, in some embodiments of the present application, step S23 may further include: obtaining a third image containing the loaded cargo in the vehicle to be identified; calling the third feature extraction layer in the trained loading rate prediction network, performing feature extraction processing based on the third image, and obtaining third feature information of the third image; calling the feature fusion layer in the trained loading rate prediction network, performing fusion processing based on the first feature information, the second feature information and the third feature information, and obtaining the target feature information.

[0151] For example, the feature extraction layer of the loading rate prediction network mentioned in step S20 may further include a third feature extraction layer.

[0152] The third feature extraction layer is used to output third feature information of the third image (i.e., image features of the loaded cargo) based on the third image (i.e., image of the loaded cargo in the vehicle to be identified). The third feature extraction layer takes the third image (i.e., image of the loaded cargo in the vehicle to be identified) as input and performs one or more operations including but not limited to convolution and pooling on the third image to extract features from the third image to obtain the third feature information of the third image (i.e., image features of the loaded cargo). The third feature extraction layer can also be a convolutional neural network. For example, the CNN network (denoted as CNN3) obtained by directly removing the fully connected layer (denoted as FC) from the classification network ResNet50 can be used as the third feature extraction layer.

[0153] The third image refers to an image of cargo loaded in the vehicle to be identified).

[0154] The third feature information refers to image features obtained after performing feature extraction processing on the image of the cargo loaded in the vehicle to be identified (ie, the third image), such as convolution, pooling, and other operations.

[0155] For example, remove the last fully connected layer of the three classification ResNet50 networks to obtain CNN1, CNN2, and CNN3.

[0156] First, CNN1 inputs the cabin area of the vehicle to be identified (i.e., the first image) to output the convolution feature Feature1 (i.e., the first feature information) of the cabin area; CNN2 inputs the image of the appearance of the vehicle to be identified (i.e., the second image) to output the convolution feature Feature2 of the vehicle appearance; CNN3 inputs the image of the loaded cargo in the vehicle to be identified (i.e., the third image) to output the convolution feature Feature3 of the loaded cargo.

[0157] Then, Feature1, Feature2, and Feature3 are input into the feature fusion layer of the trained load rate prediction network. This layer is used to perform a series fusion process on Feature1, Feature2, and Feature3 to obtain the target feature information. The specific fusion process can be referred to the introduction and example in step S23 above and will not be repeated here.

[0158] From the above, we can see that, on the one hand, the image of the loaded cargo inside the vehicle to be identified allows the load factor prediction network to identify the load factor information for different cargo types. The vehicle model, as auxiliary information, can help distinguish the differences in load factors for different cargo types. By further combining the image of the loaded cargo inside the vehicle to be identified (i.e., the third image) for load factor prediction, the recognition accuracy of the vehicle load factor is improved. On the other hand, the overall combination of the first, second, and third images enhances the feature expression ability of the load factor recognition scenario, thereby further improving the recognition accuracy of the vehicle load factor to a certain extent.

[0159] In some embodiments of the present application, the load rate prediction network is trained through the following steps A1 to A5, wherein:

[0160] A1. Obtain a sample image.

[0161] The sample images include a vehicle compartment area image of the sample vehicle and an appearance image of the sample vehicle.

[0162] The sample image has been marked with the actual loading rate and actual vehicle model information of the sample vehicle. The specific method of obtaining the sample image can refer to the introduction in the above step S10 and will not be repeated here.

[0163] A2. Calling the first feature extraction layer in the preset loading rate prediction network, performing feature extraction processing based on the carriage area image, and obtaining fourth feature information of the carriage area image.

[0164] Among them, the fourth feature information refers to the image features obtained after feature extraction processing, such as convolution and pooling, on the cabin area image of the sample vehicle during the model training stage.

[0165] Corresponding to the trained load rate prediction network, in some embodiments of the present application, the preset load rate prediction network includes a feature extraction layer and a prediction layer. The feature extraction layer may further include a first feature extraction layer and a second feature extraction layer. The functions of the first feature extraction layer, the second feature extraction layer, and the prediction layer, as well as the network input and output, can be referred to as described in step S20 above and will not be repeated here.

[0166] For example, the cabin area image of truck B is input into a preset loading rate prediction network to call the first feature extraction layer in the preset loading rate prediction network, and one or more operations such as convolution and pooling are performed on the cabin area image of truck B to achieve feature extraction of the cabin area image of truck B and obtain the image features of the cabin area image of truck B (i.e., the fourth feature information).

[0167] A3. Calling the second feature extraction layer in the preset loading rate prediction network to perform feature extraction processing based on the appearance image to obtain fifth feature information of the appearance image.

[0168] Among them, the fifth feature information refers to the image features obtained after feature extraction processing, such as convolution and pooling, on the appearance image of the sample vehicle during the model training stage.

[0169] For example, the appearance image of truck B is input into the preset loading rate prediction network to call the second feature extraction layer in the preset loading rate prediction network, and one or more operations such as convolution and pooling are performed on the appearance image of truck B to realize feature extraction of the appearance image of truck B and obtain the image features of the appearance image of truck B (i.e., the fifth feature information).

[0170] A4. Calling a feature fusion layer in a preset loading rate prediction network to perform fusion processing based on the fourth feature information and the fifth feature information to obtain image feature information of the sample image.

[0171] The image feature information refers to the image feature obtained after fusing the fourth feature information and the fifth feature information.

[0172] For ease of understanding, let's continue with the example of steps A2 and A3 above. For example, the image features of the cabin area image of truck B (denoted as feature1, with a size of [1, 2048]) and the image features of the exterior image of truck B (denoted as feature2, with a size of [1, 2048]) are input into the feature fusion layer of the preset loading rate prediction network. The feature fusion layer in the preset loading rate prediction network is called to concatenate and fuse the image features of the cabin area image of truck B (feature1) and the image features of the exterior image of truck B (feature2) into features of size [1, 4096], thereby obtaining the image feature information of the sample image.

[0173] Furthermore, to improve the accuracy of the vehicle loading rate prediction network's prediction of vehicle loading rates, in some embodiments, the sample image also includes an image of the cargo within the sample vehicle. Accordingly, the feature extraction layer of the preset loading rate prediction network may further include a third feature extraction layer. The third feature extraction layer is used to perform feature extraction processing on the cargo image to obtain sixth feature information (the sixth refers to the image features obtained after performing feature extraction processing, such as convolution and pooling, on the cargo image within the sample vehicle during the model training phase). At this point, the feature fusion layer in the preset loading rate prediction network is called to fuse the fourth feature information, the fifth feature information, and the sixth feature information to obtain image feature information of the sample image. The execution of "performing feature extraction processing on the cargo image by the third feature extraction layer to obtain the sixth feature information" can refer to the description and example of the aforementioned step of "performing feature extraction processing on the third image to obtain the third feature information of the third image" and will not be repeated here.

[0174] A5. Calling a prediction layer in a preset loading rate prediction network, performing prediction processing based on the image feature information, and obtaining the vehicle type information and the second loading rate corresponding to the sample image.

[0175] The second loading rate refers to the loading rate of the sample vehicle obtained after loading rate prediction processing is performed based on image feature information.

[0176] In some embodiments, step A5 may specifically include: performing vehicle model prediction processing based on the image feature information to obtain a second confidence level of a preset vehicle model category; performing loading rate prediction processing based on the image feature information to obtain a third confidence level of a preset loading rate category; obtaining the vehicle model category with the highest second confidence level as the vehicle model information; obtaining the loading rate category with the highest third confidence level as the second loading rate.

[0177] The second confidence level refers to the confidence level of a preset vehicle type category obtained by performing vehicle type prediction processing based on image feature information.

[0178] The third confidence level refers to the confidence level of a preset loading rate category obtained by performing loading rate prediction processing based on image feature information.

[0179] Corresponding to the trained loading rate prediction network, in some embodiments, the number of output nodes of the prediction layer in the preset loading rate prediction network is set to N, wherein N1 output nodes correspond to outputting the confidence of preset vehicle type categories (such as small, medium and large vehicle types), and N2 output nodes correspond to outputting the confidence of preset loading rate categories (such as 0%, 5%, 10%,..., 100% with an interval of 5%, a total of 21 loading rates), N, N1, and N2 are all positive integers, and N=N1+N2.

[0180] The prediction layer in the preset loading rate prediction network performs vehicle model prediction processing and loading rate prediction processing according to the target feature information, determines the second confidence level of each preset vehicle model category and the third confidence level of each preset loading rate category, and outputs them to the corresponding output node.

[0181] For example, the first three output nodes of the prediction layer represent the vehicle type categories: small, medium, and large. The next 21 nodes represent the load factor categories: 0%, 5%, 10%, ..., 100%, for a total of 21 load factors. The first three nodes output the confidence levels for the vehicle type being small, medium, and large: 0.6, 0.8, and 0.2, respectively. The fourth through 23rd nodes output the first confidence levels for the load factors being 0%, 5%, 10%, ..., and 100%, respectively: 0.2, 0.9, 0.5, ..., 0.1.

[0182] Thus, the vehicle type category "medium" with the second highest confidence level (0.8) can be obtained, that is, the vehicle type information predicted by the sample image can be obtained. The loading rate category "5%" with the third highest confidence level (0.9) can be obtained, that is, the second loading rate predicted by the sample image can be obtained.

[0183] From the above, it can be seen that by using the vehicle type category with the second highest confidence as the vehicle type prediction result of the preset load rate prediction network, the load rate prediction network's prediction accuracy for vehicle type information can be improved, thereby enhancing the characteristic expression ability of vehicle type information. By using the loading rate category with the third highest confidence as the loading rate prediction result of the preset load rate prediction network, the loading rate prediction network's prediction accuracy for loading rate can be improved.

[0184] A6. Train the preset loading rate prediction network based on the vehicle model information and the second loading rate to obtain the trained loading rate prediction network.

[0185] Specifically, the model loss value of a preset load factor prediction network is first determined based on the predicted model information and the actual model information of the sample image. The load factor loss value of the preset load factor prediction network is then determined based on the predicted second load factor of the sample image and the actual load factor. Finally, the total loss value of the preset load factor prediction network is determined based on the model loss value and the load factor loss value. Then, based on the total loss value, the model parameters of the preset load factor prediction network are updated until the preset load factor prediction network converges, resulting in a trained load factor prediction network. At this point, the load factor prediction network can predict the vehicle's load factor based on images of the vehicle's cabin area and exterior.

[0186] As can be seen from the above, the model is iteratively trained using sample images (including images of the vehicle's cabin area and exterior appearance) as input features, and the corresponding sample vehicle's vehicle model information and load factor as the desired output features until the iteration stops, resulting in a trained load factor prediction network. Because the training combines vehicle cabin area images with vehicle exterior images, the load factor prediction network can fully learn the relationship between the image features of both the vehicle's cabin area and exterior images and the vehicle's load factor, thereby enhancing the network's ability to express features for load factor recognition scenarios and, in turn, improving the network's accuracy in predicting vehicle load factors.

[0187] In addition, in order to verify the effect of the vehicle loading rate identification method provided in the embodiment of the present application, an experimental test was conducted on the solution provided in the embodiment of the present application, as follows:

[0188] The experiment selected a total of 7782 data from a transfer station over 7 days for testing. The 7782 data included 7782 vehicle trips. The basic scheme accurately predicted 6630 vehicles and incorrectly predicted 1152 vehicles; the new scheme (based on) accurately predicted 7291 vehicles and incorrectly predicted 491 vehicles.

[0189] The basic solution is to use a ResNet50 network, with a single RGB image of the carriage as input and a total of 21 output nodes (the loading rate is divided into 5% granularity [0%, 5%, 10%, ..., 100%], and recorded in order as [0, 1, 2, 3, ..., 19, 20]). The category with the highest confidence is the carriage loading rate category, and multiplying it by 5% is the carriage loading rate.

[0190] The new solution is: use two ResNet50 networks CNN1 and CNN2, and remove the last fully connected layer (FC layer) of both ResNet50 networks. CNN1 inputs the car picture and outputs the car convolution feature Feature1; CNN2 inputs the vehicle appearance picture and outputs the vehicle appearance convolution feature Feature2. Feature1 and Feature2 are fused in series and input into the prediction layer for prediction to obtain the final result. The prediction layer outputs a total of 24 nodes (the first 3 nodes are vehicle model information, representing small, medium and large models respectively, and the last 21 nodes are [0%, 5%, 10%, ..., 100%] loading rates divided according to a granularity of 5%, and are recorded in order as [0, 1, 2, 3, ..., 19, 20]). By analyzing the output results, the vehicle model information and loading rate information can be obtained.

[0191] It can be seen that the vehicle loading rate identification method provided in the embodiment of the present application improves the identification accuracy to a certain extent.

[0192] In order to better implement the vehicle loading rate identification method in the embodiment of the present application, based on the vehicle loading rate identification method, the embodiment of the present application also provides a vehicle loading rate identification device, such as Figure 7 FIG. 1 is a schematic diagram of the structure of an embodiment of a vehicle loading rate identification device in an embodiment of the present application. The vehicle loading rate identification device 700 includes:

[0193] An acquisition unit 701 is configured to acquire a first image of a vehicle compartment area including a vehicle to be identified, and acquire a second image including an appearance of the vehicle to be identified;

[0194] A feature extraction unit 702 is configured to perform feature extraction processing based on the first image and the second image to obtain target feature information of the vehicle to be identified;

[0195] The prediction unit 703 is configured to perform loading rate prediction processing based on the target feature information to obtain a first loading rate of the vehicle to be identified.

[0196] In some embodiments of the present application, the feature extraction unit 702 is further configured to:

[0197] calling a first feature extraction layer in the trained loading rate prediction network to perform feature extraction processing on the first image to obtain first feature information of the first image;

[0198] calling a second feature extraction layer in the trained loading rate prediction network to perform feature extraction processing on the second image to obtain second feature information of the second image;

[0199] The feature fusion layer in the trained loading rate prediction network is called to perform fusion processing according to the first feature information and the second feature information to obtain the target feature information.

[0200] In some embodiments of the present application, a third image is further included, and the feature extraction unit 702 is further configured to:

[0201] acquiring a third image containing the cargo loaded in the vehicle to be identified;

[0202] calling a third feature extraction layer in the trained loading rate prediction network to perform feature extraction processing on the third image to obtain third feature information of the third image;

[0203] The feature fusion layer in the trained loading rate prediction network is called to perform fusion processing according to the first feature information, the second feature information, and the third feature information to obtain the target feature information.

[0204] In some embodiments of the present application, the prediction unit 703 is further configured to:

[0205] Calling the prediction layer in the trained load rate prediction network to perform prediction processing based on the target feature information to obtain a first confidence level of a preset load rate category;

[0206] Obtaining a target loading rate category having the maximum first confidence level from the loading rate categories;

[0207] A first loading rate of the vehicle to be identified is determined according to the target loading rate category.

[0208] In some embodiments of the present application, the prediction unit 703 is further configured to:

[0209] According to the target loading rate category and the preset loading rate interval, the first loading rate of the vehicle to be identified is determined, wherein the loading rate category has N and the loading rate interval is The loading rate category is represented by a numerical value M;

[0210] The determining, according to the target loading rate category and a preset loading rate interval, a first loading rate of the vehicle to be identified includes:

[0211] The first loading rate is determined according to the following formula:

[0212]

[0213] Among them, Y represents the first loading rate, N represents the number of the loading rate categories, and M0 represents the representative value of the target loading rate category.

[0214] In some embodiments of the present application, the vehicle loading rate identification device 700 further includes a training unit (not shown in the figure), which is specifically configured to:

[0215] Acquiring a sample image, wherein the sample image includes a vehicle compartment area image of the sample vehicle and an appearance image of the sample vehicle;

[0216] calling a first feature extraction layer in a preset loading rate prediction network to perform feature extraction processing on the carriage area image to obtain fourth feature information of the carriage area image;

[0217] calling a second feature extraction layer in a preset loading rate prediction network to perform feature extraction processing on the appearance image to obtain fifth feature information of the appearance image;

[0218] calling a feature fusion layer in a preset loading rate prediction network to perform fusion processing based on the fourth feature information and the fifth feature information to obtain image feature information of the sample image;

[0219] Calling a prediction layer in a preset load rate prediction network to perform prediction processing based on the image feature information to obtain vehicle type information corresponding to the sample image and a second load rate;

[0220] The preset loading rate prediction network is trained according to the vehicle type information and the second loading rate to obtain the trained loading rate prediction network.

[0221] In some embodiments of the present application, the training unit is further configured to:

[0222] Performing vehicle type prediction processing based on the image feature information to obtain a second confidence level of a preset vehicle type category; performing loading rate prediction processing based on the image feature information to obtain a third confidence level of a preset loading rate category;

[0223] Obtain the vehicle type category with the second highest confidence as the vehicle type information; obtain the loading rate category with the third highest confidence as the second loading rate category.

[0224] In specific implementation, the above units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units can be found in the previous method embodiments and will not be repeated here.

[0225] Since the vehicle loading rate identification device can perform the following steps in the present application: Figures 1 to 6 Corresponding to the steps in the vehicle loading rate identification method in any embodiment, the present application can be implemented as follows: Figures 1 to 6The beneficial effects that can be achieved by the vehicle loading rate identification method in any embodiment are detailed in the previous description and will not be repeated here.

[0226] In addition, in order to better implement the vehicle loading rate identification method in the embodiment of the present application, based on the vehicle loading rate identification method, the embodiment of the present application also provides an electronic device, referring to Figure 8 , Figure 8 A schematic diagram of the structure of an electronic device according to an embodiment of the present application is shown. Specifically, the electronic device provided by the embodiment of the present application includes a processor 801, and the processor 801 is used to execute the computer program stored in the memory 802 to implement the following Figures 1 to 6 Corresponding to each step of the vehicle loading rate identification method in any embodiment; or, when the processor 801 is used to execute the computer program stored in the memory 802, the following is implemented Figure 7 The functions of each unit in the corresponding embodiment.

[0227] For example, the computer program may be divided into one or more modules / units, one or more of which are stored in the memory 802 and executed by the processor 801 to implement the embodiments of the present application. One or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in a computer device.

[0228] The electronic device may include, but is not limited to, a processor 801 and a memory 802. Those skilled in the art will appreciate that the illustrations are merely examples of electronic devices and do not limit the electronic device. The electronic device may include more or fewer components than shown, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, and buses, and the processor 801, memory 802, input and output devices, and network access devices are connected via a bus.

[0229] The processor 801 may be a central processing unit (CPU), or other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor. The processor is the control center of the electronic device and connects various parts of the entire electronic device using various interfaces and lines.

[0230] The memory 802 can be used to store computer programs and / or modules. The processor 801 implements various functions of the computer device by running or executing the computer programs and / or modules stored in the memory 802 and accessing the data stored in the memory 802. The memory 802 may mainly include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function (such as a sound playback function, an image playback function, etc.); the data storage area may store data generated based on the use of the electronic device (such as audio data, video data, etc.). In addition, the memory may include high-speed random access memory and non-volatile memory, such as a hard disk, internal memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0231] Those skilled in the art will clearly understand that for the convenience and brevity of description, the specific working process of the vehicle loading rate identification device, electronic device and its corresponding units described above can refer to the following. Figures 1 to 6 The description of the vehicle loading rate identification method in any embodiment will not be repeated here.

[0232] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0233] To this end, an embodiment of the present application provides a computer-readable storage medium, which stores multiple instructions, which can be loaded by a processor to execute the present application as follows: Figures 1 to 6 For the steps in the vehicle loading rate identification method in any embodiment, the specific operations can be referred to as follows: Figures 1 to 6 The description of the vehicle loading rate identification method in any embodiment will not be repeated here.

[0234] The computer-readable storage medium may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0235] Due to the instructions stored in the computer readable storage medium, the present application can be executed as follows: Figures 1 to 6 Corresponding to the steps in the vehicle loading rate identification method in any embodiment, the present application can be implemented as follows: Figures 1 to 6The beneficial effects that can be achieved by the vehicle loading rate identification method in any embodiment are detailed in the previous description and will not be repeated here.

[0236] The above is a detailed introduction to a vehicle loading rate identification method, device, electronic device and computer-readable storage medium provided in the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A vehicle loading rate identification method, characterized in that: The method comprises: Acquire a first image containing a vehicle compartment area of a vehicle to be identified, and acquire a second image containing an appearance of the vehicle to be identified; Performing feature extraction processing based on the first image and the second image to obtain target feature information of the vehicle to be identified; Performing a loading rate prediction process based on the target feature information to obtain a first loading rate of the vehicle to be identified; The performing loading rate prediction processing according to the target feature information to obtain a first loading rate of the vehicle to be identified includes: Performing prediction processing based on the target feature information to obtain a first confidence level of a preset loading rate category; Obtaining a target loading rate category having the maximum first confidence level from the loading rate categories; A first loading rate of the vehicle to be identified is determined according to the target loading rate category.

2. The vehicle loading rate identification method according to claim 1, characterized in that: The performing feature extraction processing based on the first image and the second image to obtain target feature information of the vehicle to be identified includes: calling a first feature extraction layer in the trained loading rate prediction network to perform feature extraction processing on the first image to obtain first feature information of the first image; calling a second feature extraction layer in the trained loading rate prediction network to perform feature extraction processing on the second image to obtain second feature information of the second image; The feature fusion layer in the trained loading rate prediction network is called to perform fusion processing according to the first feature information and the second feature information to obtain the target feature information.

3. The vehicle loading rate identification method according to claim 2, characterized in that: Also included is a third image, wherein the feature fusion layer in the trained loading rate prediction network is called to perform fusion processing based on the first feature information and the second feature information to obtain the target feature information, including: acquiring a third image containing the cargo loaded in the vehicle to be identified; calling a third feature extraction layer in the trained loading rate prediction network to perform feature extraction processing on the third image to obtain third feature information of the third image; The feature fusion layer in the trained loading rate prediction network is called to perform fusion processing according to the first feature information, the second feature information, and the third feature information to obtain the target feature information.

4. The vehicle loading rate identification method according to claim 2 or 3, characterized in that: The performing prediction processing according to the target feature information to obtain a first confidence level of a preset loading rate category includes: The prediction layer in the trained loading rate prediction network is called to perform prediction processing according to the target feature information to obtain a first confidence level of a preset loading rate category.

5. The vehicle loading rate identification method according to claim 4, characterized in that: The determining the first loading rate of the vehicle to be identified according to the target loading rate category includes: According to the target loading rate category and the preset loading rate interval, the first loading rate of the vehicle to be identified is determined, wherein the loading rate category has N and the loading rate interval is The loading rate category is represented by a numerical value M; The determining, according to the target loading rate category and a preset loading rate interval, a first loading rate of the vehicle to be identified includes: The first loading rate is determined according to the following formula: Among them, Y represents the first loading rate, N represents the number of the loading rate categories, and M0 represents the representative value of the target loading rate category.

6. The vehicle loading rate identification method according to claim 2, characterized in that: The method further comprises: Acquiring a sample image, wherein the sample image includes a vehicle compartment area image of the sample vehicle and an appearance image of the sample vehicle; calling a first feature extraction layer in a preset loading rate prediction network to perform feature extraction processing on the carriage area image to obtain fourth feature information of the carriage area image; calling a second feature extraction layer in a preset loading rate prediction network to perform feature extraction processing on the appearance image to obtain fifth feature information of the appearance image; calling a feature fusion layer in a preset loading rate prediction network to perform fusion processing based on the fourth feature information and the fifth feature information to obtain image feature information of the sample image; Calling a prediction layer in a preset load rate prediction network to perform prediction processing based on the image feature information to obtain vehicle type information corresponding to the sample image and a second load rate; The preset loading rate prediction network is trained according to the vehicle type information and the second loading rate to obtain the trained loading rate prediction network.

7. The vehicle loading rate identification method according to claim 6, characterized in that: The performing prediction processing based on the image feature information to obtain the vehicle type information and the second loading rate corresponding to the sample image includes: Performing vehicle type prediction processing based on the image feature information to obtain a second confidence level of a preset vehicle type category; performing loading rate prediction processing based on the image feature information to obtain a third confidence level of a preset loading rate category; The vehicle type category with the second highest confidence is obtained as the vehicle type information; and the loading rate category with the third highest confidence is obtained as the second loading rate.

8. A vehicle loading rate identification device, characterized in that: The vehicle loading rate identification device includes: an acquisition unit, configured to acquire a first image of a vehicle compartment area including a vehicle to be identified, and acquire a second image including an appearance of the vehicle to be identified; a feature extraction unit, configured to perform feature extraction processing based on the first image and the second image to obtain target feature information of the vehicle to be identified; a prediction unit, configured to perform a loading rate prediction process based on the target feature information to obtain a first loading rate of the vehicle to be identified; The prediction unit is configured to perform loading rate prediction processing based on the target feature information to obtain a first loading rate of the vehicle to be identified, including: Performing prediction processing based on the target feature information to obtain a first confidence level of a preset loading rate category; Obtaining a target loading rate category having the maximum first confidence level from the loading rate categories; A first loading rate of the vehicle to be identified is determined according to the target loading rate category.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the method for identifying a vehicle loading rate according to any one of claims 1 to 7 is executed.

10. A computer-readable storage medium, characterized in that A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in the vehicle loading rate identification method according to any one of claims 1 to 7.

Images