Method and device for determining vehicle loading rate and vehicle management system

Abstract

Embodiments of the present invention provide a method, device, and vehicle management system for determining a vehicle's loading rate. The method comprises: obtaining an image of a vehicle compartment loaded with cargo, wherein the compartment image is an image captured by an image acquisition device from the rear of the vehicle toward the interior of the compartment; inputting the compartment image into a depth estimation model, and obtaining the distance from the cargo side of the vehicle to the image acquisition device through monocular depth estimation; and determining the vehicle's loading rate based on the distance from the cargo side to the image acquisition device. Monocular depth estimation is performed using the depth estimation model to determine the distance from the cargo side of the vehicle to the image acquisition device, and the vehicle's loading rate can then be determined based on the distance from the cargo side to the image acquisition device. Because monocular depth estimation eliminates the need for complex depth information acquisition equipment, the vehicle's loading rate can be accurately and cost-effectively determined.

Description

Technical Field

[0001] The present invention relates to the technical field of logistics management, and in particular to a method and device for determining a vehicle loading rate, and a vehicle management system. Background Art

[0002] As a place for loading and unloading vehicles, docking stations are an essential component of cargo flow. Visualization and digital management of docking stations have become a key focus in the development of smart logistics parks. One crucial piece of data is determining the vehicle loading rate during the loading and unloading process. With the rapid development of deep learning and video image processing technologies, users have placed higher demands on real-time vehicle loading rates.

[0003] Commonly used solutions for determining vehicle loading rates in logistics parks include binocular vision, multi-dimensional laser scanning, and ultrasonic probes. Most of these solutions are based on external depth sensors to obtain depth information, and then calculate the vehicle loading rate based on the distance from the cargo surface to the depth sensor.

[0004] Due to the high price of depth sensors, these solutions are costly and have limited scalability. While monocular cameras, which use pixel intersection-over-union (IoU) calculations, are cheaper, their accuracy is poor and cannot meet practical requirements. Therefore, a low-cost method that can accurately determine vehicle loading rates is urgently needed. Summary of the Invention

[0005] The purpose of the embodiments of the present invention is to provide a method, device, vehicle management system, electronic device, and storage medium for determining vehicle loading rate, so as to accurately obtain the vehicle loading rate at low cost. The specific technical solution is as follows:

[0006] In a first aspect, an embodiment of the present invention provides a method for determining a vehicle loading rate, the method comprising:

[0007] Acquire a compartment image of a vehicle loaded with cargo, wherein the compartment image is an image captured by an image acquisition device from the rear of the vehicle toward the interior of the compartment;

[0008] Inputting the vehicle compartment image into a depth estimation model, and obtaining the distance from the cargo surface of the vehicle to the image acquisition device through monocular depth estimation;

[0009] The loading rate of the vehicle is determined based on the distance from the cargo surface to the image acquisition device.

[0010] Optionally, the step of determining the loading rate of the vehicle based on the distance from the cargo surface to the image acquisition device includes:

[0011] Based on the distance between the cargo surface and the image acquisition device, an integral algorithm is used to obtain the free volume of the carriage;

[0012] The loading rate of the carriage is calculated based on the volume of the carriage and the free volume.

[0013] Optionally, the method further includes:

[0014] Acquire a platform image of the platform where the vehicle is parked;

[0015] Performing personnel recognition on the platform image to determine the number of operating personnel included in the platform image;

[0016] The loading operation efficiency of the vehicle is calculated based on the loading rate, operation time and quantity of the vehicle compartment.

[0017] Optionally, the method further includes:

[0018] Acquiring a plurality of vehicle images of the vehicle in a target state, wherein the target state includes entering a platform and / or exiting a platform;

[0019] Performing vehicle detection on the vehicle image to determine a movement trajectory of the vehicle;

[0020] Based on the movement trajectory and the occupancy status of the parking spaces on the platform, entry and exit information of the vehicle is determined.

[0021] Optionally, the step of performing vehicle detection on the vehicle image to determine the movement trajectory of the vehicle includes:

[0022] Performing vehicle recognition on the vehicle image to obtain a plurality of vehicle target recognition results, wherein the vehicle target recognition results include vehicle positions;

[0023] Performing license plate recognition on the vehicle image to obtain multiple license plate target recognition results;

[0024] Associating and matching the vehicle target recognition result with the license plate target recognition result to obtain a detection result queue;

[0025] The movement trajectory of the vehicle is determined based on the acquisition time and vehicle position corresponding to the matching vehicle target recognition result and the license plate target recognition result in the detection result queue.

[0026] Optionally, the target state is entering the platform;

[0027] The step of determining the vehicle entry and exit information based on the motion trajectory and the occupancy status of the parking spaces on the platform includes:

[0028] Obtaining the occupancy status of the parking spaces on the platform and determining whether the motion trajectory disappears;

[0029] If the motion trajectory disappears and the occupancy status of the parking space on the platform changes from idle to occupied, determining that the vehicle's entry and exit information is normal entry;

[0030] If the motion trajectory disappears and the occupied state of the parking space on the platform remains vacant, determining that the entry and exit information of the vehicle is an abnormal entry;

[0031] If the motion trajectory disappears and the occupancy state of the parking space on the platform remains occupied, when the original occupancy state of the parking space is a blocked state, determining that the entry and exit information of the vehicle is a normal entry;

[0032] If the motion track disappears and the occupancy status of the parking space on the platform remains occupied, when the original occupancy status of the parking space is an existing vehicle status, the entry and exit information of the vehicle is determined to be abnormal entry.

[0033] Optionally, the target state is driving out of the platform;

[0034] The step of determining the vehicle entry and exit information based on the motion trajectory and the occupancy status of the parking spaces on the platform includes:

[0035] Obtaining the occupancy status of the parking spaces on the platform and determining whether the motion trajectory disappears;

[0036] If the motion trajectory disappears and the occupancy status of the parking space on the platform changes from occupied to free, it is determined that the entry and exit information of the vehicle is normal exit;

[0037] If the motion track disappears and the occupancy status of the parking space on the platform remains occupied, it is determined that the entry and exit information of the vehicle is abnormal.

[0038] Optionally, the method further includes:

[0039] Performing door detection on the vehicle image to determine the open / close state of the vehicle compartment door;

[0040] If the target state is entering the platform and the switch state is closed, output first door abnormality information;

[0041] If the target state is to leave the platform and the switch state is open, the second door abnormality information is output.

[0042] Optionally, the method further comprises at least one of the following steps:

[0043] If the target state is entering the platform, pedestrian detection is performed on the vehicle image, and if the reversing area of ​​the vehicle image includes a pedestrian, a first alarm message is output; or

[0044] If the target state is entering the platform, performing vehicle position detection on the vehicle image, and outputting a second alarm message if the vehicle position does not conform to the parking position; or

[0045] If the target state is to leave the platform and the loading rate of the carriage is less than a preset threshold, a third alarm message is output.

[0046] In a second aspect, an embodiment of the present invention provides a device for determining a vehicle loading rate, the device comprising:

[0047] A compartment image acquisition module, configured to acquire an image of a compartment of a vehicle loaded with cargo, wherein the compartment image is an image acquired by an image acquisition device from the rear of the vehicle toward the interior of the compartment;

[0048] a distance calculation module, configured to input the vehicle compartment image into a depth estimation model and obtain the distance from the cargo side of the vehicle to the image acquisition device through monocular depth estimation;

[0049] The loading rate determination module is used to determine the loading rate of the vehicle based on the distance from the cargo surface to the image acquisition device.

[0050] In a third aspect, an embodiment of the present invention provides a vehicle management system, the system including an image acquisition device and an integrated management platform, wherein:

[0051] The image acquisition device is used to perform any of the method steps of the first aspect above;

[0052] The integrated management platform is used to obtain the platform information reported by the image acquisition device, wherein the platform information includes at least one of image information, vehicle information, parking space occupancy status, and alarm information.

[0053] Optionally, the system further includes a platform management platform;

[0054] The platform management platform is used to obtain the platform information stored in the integrated management platform and the parking space reservation information sent by the reservation system, and complete the parking space reservation based on the platform information and the reservation information.

[0055] Optionally, the system further includes a display screen;

[0056] The platform management platform is further used to control the display screen to display the platform status on the electronic map based on the parking space reservation information, the vehicle information and the occupancy status of the parking space.

[0057] Optionally, the system further includes a fill light;

[0058] The integrated management platform is further used to control the fill light to provide fill light for the image acquisition device based on the ambient light intensity.

[0059] In a fourth aspect, an embodiment of the present invention provides an electronic device, comprising a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0060] Memory for storing computer programs;

[0061] The processor is configured to implement any of the method steps described in the first aspect when executing a program stored in the memory.

[0062] In a fifth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of the first aspects are implemented.

[0063] Beneficial effects of the embodiments of the present invention:

[0064] In the solution provided by an embodiment of the present invention, an electronic device can obtain an image of the compartment of a vehicle loaded with cargo, wherein the compartment image is an image captured by an image acquisition device from the rear of the vehicle toward the interior of the compartment; and the compartment image is input into a depth estimation model. Through monocular depth estimation, the electronic device can obtain the distance from the cargo side of the vehicle to the image acquisition device; the electronic device can also determine the vehicle's loading rate based on the distance from the cargo side to the image acquisition device. The electronic device can perform monocular depth estimation using the depth estimation model to determine the distance from the cargo side of the vehicle to the image acquisition device, and further determine the vehicle's loading rate based on the distance from the cargo side to the image acquisition device. Since monocular depth estimation eliminates the need to set up complex depth information acquisition equipment, the vehicle's loading rate can be accurately and at a low cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0066] Figure 1 A flow chart of a method for determining a vehicle loading rate provided by an embodiment of the present invention;

[0067] Figure 2 Based on Figure 1 A schematic diagram of the installation position of the image acquisition device of the illustrated embodiment;

[0068] Figure 3 Based on Figure 1 A schematic diagram of an image of cargo inside a carriage captured by the image acquisition device of the illustrated embodiment;

[0069] Figure 4(a) is based on Figure 1 A schematic diagram of a carriage image of the illustrated embodiment;

[0070] Figure 4(b) is based on Figure 1 A schematic diagram of a depth diagram of the cargo side of a carriage in the illustrated embodiment;

[0071] Figure 5 for Figure 1 A specific flow chart of step S103 in the embodiment shown;

[0072] Figure 6 Based on Figure 1 A flow chart of a method for calculating loading operation efficiency according to the illustrated embodiment;

[0073] Figure 7 Based on Figure 1 A flow chart of determining a vehicle's loading rate and calculating working status and efficiency according to the illustrated embodiment;

[0074] Figure 8 Based on Figure 1 A flow chart of a method for determining vehicle entry and exit information according to the illustrated embodiment;

[0075] Figure 9 Based on Figure 1 A schematic diagram of the process of a vehicle entering a platform according to the embodiment shown;

[0076] Figure 10 for Figure 8 A specific flow chart of step S802 in the embodiment shown;

[0077] Figure 11 for Figure 8 A specific flow chart of step S803 in the embodiment shown;

[0078] Figure 12 for Figure 8 Another specific flow chart of step S803 in the illustrated embodiment;

[0079] Figure 13 Based on Figure 8 A flow chart of a method for outputting door abnormality information according to the illustrated embodiment;

[0080] Figure 14 Based on Figure 1 A specific flow chart of the method for determining the loading rate of a vehicle according to the illustrated embodiment;

[0081] Figure 15 A schematic structural diagram of a vehicle loading rate determination device provided by an embodiment of the present invention;

[0082] Figure 16 A schematic diagram of a vehicle management system provided by an embodiment of the present invention;

[0083] Figure 17 Another schematic diagram of a vehicle management system provided by an embodiment of the present invention;

[0084] Figure 18 The present invention provides a schematic structural diagram of an electronic device. DETAILED DESCRIPTION

[0085] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field based on the present invention are within the scope of protection of the present invention.

[0086] In order to obtain the vehicle loading rate at low cost and accurately, an embodiment of the present invention provides a method, device, vehicle management system, image acquisition equipment, computer-readable storage medium and computer program product for determining the vehicle loading rate. The following first introduces a method for determining the vehicle loading rate provided by an embodiment of the present invention.

[0087] The vehicle loading factor determination method provided in the embodiments of the present invention can be applied to any electronic device that needs to determine the vehicle loading factor, such as a camera installed at a platform, a server in a logistics park, or information management equipment, without specific limitation herein. For clarity, the electronic device will be referred to as such.

[0088] like Figure 1 As shown, a method for determining a vehicle loading rate comprises:

[0089] S101, acquiring a compartment image of a vehicle loaded with cargo;

[0090] The vehicle compartment image is an image captured by an image acquisition device from the rear of the vehicle toward the interior of the vehicle compartment.

[0091] S102: Input the vehicle compartment image into a depth estimation model, and obtain the distance from the cargo surface of the vehicle to the image acquisition device through monocular depth estimation.

[0092] S103: Determine the loading rate of the vehicle based on the distance from the cargo surface to the image acquisition device.

[0093] It can be seen that in the solution provided by the embodiment of the present invention, the electronic device can obtain an image of the compartment of a vehicle loaded with cargo, wherein the compartment image is an image captured by the image acquisition device from the rear of the vehicle toward the interior of the compartment; and the compartment image is input into the depth estimation model. Through monocular depth estimation, the electronic device can obtain the distance from the cargo side of the vehicle to the image acquisition device; the electronic device can also determine the loading rate of the vehicle based on the distance from the cargo side to the image acquisition device. The electronic device can perform monocular depth estimation through the depth estimation model to determine the distance from the cargo side of the vehicle to the image acquisition device, and then determine the loading rate of the vehicle based on the distance from the cargo side to the image acquisition device. Since the use of monocular depth estimation does not require the installation of complex depth information acquisition equipment, the loading rate of the vehicle can be obtained at low cost and accurately.

[0094] As a place for loading and unloading vehicles, docks are an essential component of cargo flow. Visualization and digital management of docks have become a key focus in the development of smart logistics parks. One crucial piece of data is the load factor of vehicles during loading and unloading. To improve dock operational efficiency and reduce vehicle operating costs, users have placed higher demands on real-time access to vehicle load factors.

[0095] The platform is usually located outside the warehouse in the logistics park, and has at least one parking space. In order to visually manage the platform and obtain vehicle-related information, an image acquisition device is installed in the scene where the platform is located. The image acquisition device can be installed at the top of the platform and the warehouse. In this way, the image acquisition device has a better viewing angle and can obtain images including license plates, parking space information, etc. Figure 2 As shown, the image acquisition device 201 can be specifically an IPC (Internet Protocol Camera), which is installed at the platform and the top of the warehouse. In this way, during the process of loading cargo on the vehicle, the IPC can capture images of the cargo inside the vehicle, for example, Figure 3 shown.

[0096] During the cargo loading process, in order to accurately determine the loading rate of the vehicle, in the above step S101, the electronic device can obtain an image of the vehicle compartment loaded with cargo, wherein the image of the vehicle compartment is an image captured by the image acquisition device from the rear of the vehicle toward the interior of the vehicle compartment, for example, the image acquisition device can be captured from the vehicle compartment. Figure 2 The car compartment image is captured from the perspective shown.

[0097] Next, the electronic device may input the acquired vehicle compartment image into a depth estimation model to obtain the distance from the cargo side of the vehicle to the image acquisition device through monocular depth estimation, that is, executing the above step S102.

[0098] The depth estimation model can be a pre-trained deep learning model such as a convolutional neural network or a recurrent neural network, and is not specifically limited here. When training the depth estimation model, multiple images of the carriage with different cargo loading and unloading volumes are collected as sample images, and the depth of the cargo surface in the sample images is calibrated to obtain a label corresponding to each sample image. The sample images are input into the depth estimation model, and the depth estimation model extracts features from the sample images based on the current parameters. Based on the extracted features, the distance from the cargo surface to the image acquisition device is determined as a prediction result, and the prediction result is output. Furthermore, the parameters of the depth estimation model are continuously adjusted based on the difference between the label corresponding to the sample image and the prediction result until the depth estimation model converges and training is completed. At this point, the distance from the cargo surface to the image acquisition device output by the depth estimation model is accurate and can achieve the accuracy required in practice.

[0099] In this way, the vehicle compartment image is input into the depth estimation model, and the depth estimation model can extract the features of the compartment image, and then determine the distance from the cargo surface to the image acquisition device based on the features, and output the distance.

[0100] For example, after the car compartment image 410 shown in Figure 4(a) is input into the depth estimation model, the depth estimation model can extract the features of the car compartment image 410, and then determine the distance from the cargo surface in the car compartment image 410 to the image acquisition device based on the features, such as the depth diagram 420 of the cargo surface shown in Figure 4(b).

[0101] After determining the distance between the cargo surface and the image capture device, the electronic device may determine the vehicle's loading factor based on the distance between the cargo surface and the image capture device in step S103. In one embodiment, after determining the distance between the cargo surface and the image capture device, the electronic device may calculate the volume of the unloaded area of ​​the vehicle's compartment using an integration algorithm to determine the vehicle's loading factor.

[0102] In the solution provided by the embodiments of the present invention, an electronic device can capture an image of the cargo compartment of a loaded vehicle. Using monocular depth estimation and a depth estimation model, the electronic device can determine the distance between the cargo area of ​​the vehicle and the image acquisition device, thereby determining the vehicle's loading rate. This solution eliminates the need for a depth sensor, reducing costs and enhancing scalability. The depth obtained using the depth estimation model is highly accurate, allowing accurate determination of the vehicle's loading rate.

[0103] As an implementation method of the embodiment of the present invention, Figure 5 As shown, the step of determining the loading rate of the vehicle based on the distance from the cargo surface to the image acquisition device may include:

[0104] S501 , based on the distance from the cargo surface to the image acquisition device, an integral algorithm is used to obtain the free volume of the carriage.

[0105] After the electronic device obtains the distance from the cargo surface of the vehicle to the image acquisition device, if the length, width and height data of the vehicle's compartment are known, it can perform an integral calculation to obtain the volume of the unloaded area of ​​the compartment, which is the free volume of the compartment. In one embodiment, the cargo is stacked in the compartment from bottom to top, and the upper surface of the cargo in the compartment forms the cargo surface. The volume of the curved top cylinder formed with the spatial curved surface formed by the cargo surface of the vehicle as the top and the projection of the curved surface on the plane where the top of the compartment is located as the bottom is the free volume of the compartment. Specifically, any of the four vertices on the top of the compartment can be used as the coordinate origin, the width direction of the compartment as the X-axis direction, the height direction of the compartment as the Y-axis direction, and the length direction of the compartment as the Z-axis direction to establish a spatial rectangular coordinate system. The electronic device can obtain the three-dimensional curved surface of the cargo surface based on the distance from the cargo surface of the vehicle to the image acquisition device, that is, the surface y = f(x,z). The electronic device can also obtain the plane where the top of the car is located, that is, the XOZ plane. The cargo surface is projected onto the top plane of the car, and the projection range is the entire top plane of the car. Therefore, the projection of the surface y=f(x,z) on the XOZ plane is area D, and D is Where x0 is the width of the car and z0 is the length of the car. By calculating the double integral from the three-dimensional surface of the cargo surface to the plane where the top of the car is located, that is, calculating ∫∫ D f(x,z)dxdz, the integral value obtained is the spare volume of the carriage.

[0106] S502: Calculate the loading rate of the carriage according to the volume of the carriage and the free volume.

[0107] If the length, width, and height of the vehicle's cabin are known, the cabin's volume can be calculated. The cabin's volume can be determined and marked during the vehicle's production, or it can be calculated by multiplying the length, width, and height data. After obtaining the cabin's unoccupied volume, the electronic device can calculate the difference between the unoccupied volume and the cabin's volume based on the unoccupied volume and the cabin's volume. The ratio of this difference to the cabin's volume is then calculated as the cabin's loading rate. For example, if the vehicle's cabin is marked as 4.2 meters long, 1.9 meters wide, and 1.8 meters high, with a volume of 14 cubic meters, or if the vehicle's cabin is measured to be 4.2 meters long, 1.9 meters wide, and 1.8 meters high, the calculated cabin volume is 4.2 × 1.9 × 1.8 = 14.365 cubic meters, or approximately 14 cubic meters. Take one of the four vertices on the top of the car as the coordinate origin, the width of the car as the X-axis, the height of the car as the Y-axis, and the length of the car as the Z-axis to establish a spatial rectangular coordinate system. The cargo surface of the vehicle can be expressed as a surface y = f (x, z) in this spatial rectangular coordinate system. The projection of the surface y = f (x, z) on the XOZ plane is area D, and D is In this way, we can calculate ∫∫ D The value of f(x,z)dxdz. Assume that the electronic device calculates the empty volume of the car through this integral to be 4.2 cubic meters. In this way, the loading rate of the car can be calculated as The compartment is 70% loaded.

[0108] When the length, width and height data of the vehicle's compartment are unknown, it is difficult for electronic equipment to accurately calculate the vacant volume of the compartment. The distance from the cargo surface to the image acquisition device can be output to facilitate real-time detection of changes in the distance from the cargo surface to the image acquisition device.

[0109] As can be seen, in this embodiment, the electronic device can calculate the vehicle's loading rate by integrating the available volume based on the distance from the cargo area to the image acquisition device, given the known length, width, and height data of the vehicle compartment. This allows the electronic device to accurately and cost-effectively determine the vehicle's loading rate.

[0110] As an implementation method of the embodiment of the present invention, Figure 6 As shown, the above method may further include:

[0111] S601: Acquire a platform image of the platform where the vehicle is parked.

[0112] When a loaded vehicle enters or exits a platform, the image acquisition device can capture images in real time. In one case, the captured image is an image of the vehicle compartment, based on which the electronic device can determine the vehicle's loading rate. In another case, the image acquisition device can capture images of the platform area, including parking spaces, vehicles waiting to be parked, platform workers, and cargo. In this way, the electronic device can obtain an image of the platform where the vehicle is parked.

[0113] S602: Perform personnel recognition on the platform image to determine the number of workers included in the platform image.

[0114] To determine the number of workers, the electronic device can perform personnel recognition on the platform image. For example, a deep learning model can be used to perform personnel recognition on the platform image, thereby determining the number of workers included in the platform image. The deep learning model can be any model capable of recognizing people in an image, and is not specifically limited here.

[0115] In one embodiment, the electronic device can input multiple frames of continuous platform images into the deep learning model, and the deep learning model can perform motion detection on the workers in the platform images to determine the number of workers performing loading and unloading operations, thereby more accurately determining the number of workers included in the platform images.

[0116] S603: Calculate the loading efficiency of the vehicle according to the loading rate, operation duration, and quantity of the vehicle compartment.

[0117] The electronic device can obtain the start time and end time of the loading and unloading operation, and combine the start time and end time to obtain the operation duration. In one embodiment, after the loading and unloading operation begins, the loading rate of the carriage changes, and the electronic device can record this time as the start time; after the loading and unloading operation is completed, the loading rate no longer changes, and the electronic device can record this time as the end time; the difference between the start time and the end time is the loading and unloading operation time. The electronic device can also obtain the change in the loading rate within a certain period of time of the loading and unloading operation, and calculate the loading operation efficiency of the time period based on the change in the loading rate, the length of the time period, and the number of operators.

[0118] By combining the changes in the loading rate of the carriage, the duration of loading and unloading operations, and the number of operators, the electronic device can calculate the loading efficiency of the vehicle. First, the difference between the maximum and minimum loading rates within the loading and unloading operation time is calculated, then the product of the loading and unloading operation time and the number of operators is calculated, and finally the ratio of the difference to the product is calculated as the loading efficiency of the vehicle. For example, if the loading and unloading operation time is 5 minutes and the number of operators is 4, and during this operation time, the carriage loading rate changes from 5% to 90%, the loading efficiency can be calculated as That is, the loading capacity per person per minute is 4.25% of the carriage volume.

[0119] As can be seen, in this embodiment, the electronic device can capture an image of the dock where the vehicle is parked, thereby determining the number of workers and, combined with the vehicle loading rate and operation duration, the vehicle loading efficiency. This allows the efficiency of the dock workers to be calculated, and their work status and loading compliance to be determined, facilitating dock management during logistics operations.

[0120] In one embodiment, the electronic device can calculate the vehicle loading rate and statistical working status and efficiency based on a deep learning algorithm through a network such as monocular depth estimation, image detection and segmentation, and target classification. The process of determining the vehicle loading rate and statistical working status and efficiency is as follows: Figure 7 Shown, including:

[0121] S701, ROI (Regions Of Interest) is determined;

[0122] The area where the vehicle is located is used as ROI and as input for monocular depth estimation, image detection and segmentation, and target classification networks.

[0123] S702, monocular depth estimation;

[0124] Monocular depth estimation is performed through the depth estimation model.

[0125] S703, segmenting the cargo surface and estimating the distance;

[0126] The cargo surface is segmented from the carriage image and the distance from the cargo surface to the image acquisition device is estimated.

[0127] S704, calculating the loading rate;

[0128] The free volume and loading rate of the carriage are calculated by integration.

[0129] S705, target detection;

[0130] Perform person recognition on platform images.

[0131] S706, inspection personnel;

[0132] Determine the number of workers and their working hours in the dock image.

[0133] S707, statistics of work status and efficiency;

[0134] The vehicle's loading efficiency is calculated based on the loading rate, operation time and number of carriages.

[0135] S708, output loading rate, working status and efficiency;

[0136] Output the calculated vehicle loading rate and statistical working status and efficiency.

[0137] As can be seen, in this embodiment, the electronic device can accurately and cost-effectively determine the vehicle loading rate. It can also calculate the vehicle loading efficiency based on the number of operators, the vehicle compartment loading rate, and the operation duration. This can be used to calculate the efficiency of platform operators, determine their work status, and determine loading compliance, facilitating platform management during logistics operations.

[0138] As an implementation method of the embodiment of the present invention, Figure 8 As shown, the above method may further include:

[0139] S801, acquiring multiple vehicle images of the vehicle in a target state;

[0140] In order to manage the vehicle in a comprehensive manner, the electronic device can also obtain multiple vehicle images of the vehicle in a target state, wherein the target state may include entering the platform and / or leaving the platform. For example, the process of the vehicle entering the platform, such as Figure 9 As shown, the vehicle is a truck, including a compartment with a closable door. Before entering the platform, it is necessary to open the compartment door, point the rear of the vehicle toward the platform, and drive into the parking space on the platform to be parked.

[0141] When a vehicle enters and / or exits the platform, with its rear end facing the platform, the electronic device can clearly identify the vehicle and its license plate, and the image acquisition device can continuously capture multiple images of the vehicle. In this way, the electronic device can obtain multiple images of the vehicle in the target state.

[0142] S802: Perform vehicle detection on the vehicle image to determine a movement trajectory of the vehicle.

[0143] To determine the vehicle's trajectory, the electronic device can perform vehicle detection on multiple vehicle images. For example, a deep learning model can be used to detect multiple vehicle images and determine the vehicle's position within the images. The deep learning model can be any model capable of identifying vehicles in images, and is not specifically limited here. As the vehicle moves, its position within the images changes continuously. By combining these changes in the vehicle's position across multiple images, the electronic device can determine the vehicle's trajectory. The vehicle's trajectory can include disappearing after entering the platform or disappearing after exiting the platform.

[0144] In one implementation, the electronic device can input multiple pre-collected vehicle image samples from different time periods, weather conditions, and scenes into a deep learning model, perform appropriate cropping and localization, and optimize the model size and time consumption to achieve real-time vehicle detection. This deep learning model can then detect vehicle motion in the images and determine their trajectory.

[0145] S803: Determine the entry and exit information of the vehicle based on the movement trajectory and the occupancy status of the parking spaces on the platform.

[0146] In one embodiment, an image acquisition device captures an image of the parking space, and the electronic device can perform parking space detection on the image. A deep learning model can be used to detect vehicles within the parking space area in the image, determine whether there are vehicles within the area, and thus determine whether there is a vehicle parked in the parking space. The electronic device also records the occupancy status of the parking space. For example, the occupancy status of the parking space can be specifically "occupied," "vacant," "blocked," etc., which are not limited here. If a vehicle is detected parked, the electronic device can also record the time the vehicle parked in the parking space and the time the vehicle left the parking space.

[0147] Before the electronic device detects the movement trajectory of the vehicle, it indicates that there is no vehicle entering or exiting the vehicle, and the occupancy status of the parking space can be recorded as the original occupancy status; when the movement trajectory of the vehicle is detected, it indicates that a vehicle is entering or exiting the platform; after detecting that the movement trajectory disappears, it indicates that the vehicle's entry and exit actions have been completed, and the occupancy status of the parking space can be recorded as the current occupancy status. In this way, the electronic device can determine whether the vehicle's entry and exit process is normal based on the original occupancy status of the parking space, the vehicle's movement trajectory, and the current occupancy status of the parking space, and thereby determine the vehicle's entry and exit information. For example, the vehicle's entry and exit information can be "normal exit", "normal entry", "abnormal exit", "abnormal entry", etc., which are not limited here.

[0148] As can be seen, in this embodiment, the electronic device can capture multiple images of the vehicle in the target state and thereby determine the vehicle's motion trajectory. Based on the motion trajectory and the occupancy status of the parking spaces at the platform, the electronic device can determine the vehicle's entry and exit information. In this way, the electronic device can update the vehicle's entry and exit information in a timely and accurate manner.

[0149] As an implementation method of the embodiment of the present invention, Figure 10 As shown, the above-mentioned step of performing vehicle detection on the vehicle image and determining the movement trajectory of the vehicle may include:

[0150] S1001, performing vehicle recognition on the vehicle image to obtain multiple vehicle target recognition results;

[0151] Wherein, the vehicle target recognition result includes the vehicle position.

[0152] Using a deep learning network model, multiple vehicle images are used for vehicle recognition. The electronic device can obtain multiple vehicle target recognition results, each including the vehicle's location. Multiple vehicle images can be used to generate multiple vehicle target recognition results. The electronic device can then create a target queue of vehicle target recognition results, with each vehicle target recognition result in the queue corresponding to a vehicle location.

[0153] S1002: Perform license plate recognition on the vehicle image to obtain multiple license plate target recognition results.

[0154] Through the deep learning network model, license plate recognition is performed on multiple vehicle images. In one embodiment, based on methods such as license plate positioning, character segmentation, character recognition, including but not limited to attention (attention mechanism), CTC (Connectionist Temporal Classification based on neural network temporal classification) and other technical means, the electronic device can obtain multiple license plate target recognition results. The electronic device can also combine the matching multiple frames of adjacent license plate results to statistically calculate a license plate result with the highest confidence, and output it as the license plate target recognition result. For example, the license plate information recognized in the first five frames of vehicle images is "12345", and the license plate information recognized in the current frame is "12346", then the license plate number actually output in the current frame can be determined to be "12345" based on the multi-frame statistical information. The use of a multi-frame license plate selection scheme can effectively reduce the probability of license plate recognition errors and improve the effectiveness of the entire evidence collection result.

[0155] Multiple vehicle images can be used to identify multiple vehicle target recognition results. The electronic device can also establish a target queue of license plate target recognition results, where each license plate target recognition result in the queue corresponds to a license plate information.

[0156] S1003, associating and matching the vehicle target recognition result with the license plate target recognition result to obtain a detection result queue.

[0157] The electronic device can associate and match the target queue of the vehicle target recognition result with the target queue of the license plate target recognition result, thereby determining the matched vehicle target recognition result and license plate target recognition result to obtain the detection result queue.

[0158] S1004: Determine the movement trajectory of the vehicle based on the acquisition time and vehicle position corresponding to the matching vehicle target recognition result and the license plate target recognition result in the detection result queue.

[0159] The detection result queue includes matching vehicle target recognition results and license plate target recognition results. The vehicle positions corresponding to these multiple matching vehicle target recognition results and license plate target recognition results are different, and the corresponding acquisition times are also different. The electronic device can determine the vehicle's movement trajectory based on the acquisition time and vehicle position. For example, the electronic device can arrange the multiple matching vehicle target recognition results and license plate target recognition results in chronological order and extract the vehicle position information from the vehicle target recognition results. In this way, the electronic device obtains the vehicle position changes arranged in chronological order, thereby determining the vehicle's movement trajectory.

[0160] As can be seen, in this embodiment, the electronic device can associate and match the vehicle target recognition result with the license plate target recognition result to determine the vehicle's movement trajectory, thereby avoiding the problem of losing track of the target due to detection loss.

[0161] As an implementation method of the embodiment of the present invention, for the case where the target state is to enter the platform, such as Figure 11 As shown, the step of determining the vehicle entry and exit information based on the motion trajectory and the occupancy status of the parking spaces on the platform may include:

[0162] S1101, obtaining the occupancy status of the parking space on the platform and determining whether the motion trajectory disappears. If the motion trajectory disappears and the occupancy status of the parking space on the platform changes from idle to occupied, executing step S1102; if the motion trajectory disappears and the occupancy status of the parking space on the platform remains idle, executing step S1103; if the motion trajectory disappears and the occupancy status of the parking space on the platform remains occupied, executing step S1104; if the motion trajectory disappears and the occupancy status of the parking space on the platform remains occupied, executing step S1105;

[0163] If the motion trajectory disappears while the vehicle is entering the platform, it may mean that the vehicle has entered the platform from outside and completed the parking process at the platform; it may also mean that the vehicle has left the blind spot of the field of view or the target was lost during the vehicle detection process.

[0164] S1102, determining that the vehicle's entry and exit information is normal entry.

[0165] The motion trajectory disappears and the occupancy status of the platform parking space changes from idle to occupied, indicating that the original occupancy status of the platform parking space is idle. The vehicle enters the platform from outside and parks normally in the platform parking space. The electronic device updates the occupancy status of the parking space to occupied. It can be determined that the vehicle's entry and exit information is normal entry.

[0166] S1103: Determine that the vehicle's entry and exit information is abnormal entry.

[0167] If the motion trajectory disappears and the occupancy status of the platform parking space remains free, it means that the original occupancy status of the platform parking space is free. The electronic device detects that the occupancy status of the parking space is still free, which means that the vehicle has left the blind spot of the field of view or lost the target and did not actually enter the vehicle. The vehicle entry and exit information can be determined as abnormal entry.

[0168] S1104: When the original occupied state of the parking space is a blocked state, determining that the entry and exit information of the vehicle is a normal entry.

[0169] S1105 , when the original occupied state of the parking space is a vehicle existing state, determining that the entry and exit information of the vehicle is an abnormal vehicle entry.

[0170] The motion trajectory disappears and the occupancy status of the platform parking space remains occupied, indicating that the original occupancy status of the platform parking space is occupied. This original occupancy status indicates that there are two possible situations when the vehicle enters the platform:

[0171] In the first scenario, a parking space on the platform may be obstructed by objects such as cargo being loaded or unloaded, or platform staff. The image acquisition device can capture images of the obstructions. The electronic device can then analyze the captured images and determine that the original occupancy state of the parking space was obstructed. If a vehicle enters the platform from outside and parks normally in a parking space, the vehicle's entry and exit information can be determined to be normal.

[0172] In the second case, a vehicle may already be parked in a parking space on the platform. The image acquisition device can capture images of the parked vehicle. The electronic device can then analyze the captured images and determine that the original occupancy status of the parking space was that of an occupied vehicle. A vehicle entering the platform from outside may not have parked in the correct parking space, and the vehicle's entry and exit information can be determined to be an abnormal entry.

[0173] As can be seen, in this embodiment, when the target state is entering the platform, the electronic device can determine the vehicle's entry information based on the motion trajectory and the occupancy status of the platform's parking spaces. This solves the pain points of occupancy status errors caused by occupancy, target trajectory loss, and vehicle entry and exit in blind spots, thereby enhancing adaptability.

[0174] As an implementation method of the embodiment of the present invention, for the case where the target state is to leave the platform, such as Figure 12 As shown, the step of determining the vehicle entry and exit information based on the motion trajectory and the occupancy status of the parking spaces on the platform may include:

[0175] S1201, obtaining the occupancy status of the parking space on the platform and determining whether the motion trajectory disappears. If the motion trajectory disappears and the occupancy status of the parking space on the platform changes from occupied to free, executing step S1202; if the motion trajectory disappears and the occupancy status of the parking space on the platform remains occupied, executing step S1203;

[0176] When the vehicle is leaving the platform, if the motion track disappears, it means that the vehicle has left the parking space and left the platform.

[0177] S1202: Determine whether the vehicle's entry and exit information is normal.

[0178] The movement trajectory disappears and the occupancy status of the platform parking space changes from occupied to free, indicating that the original occupancy status of the platform parking space was occupied. The vehicle then exits the platform, and the electronic device updates the parking space's occupancy status to free. This confirms that the vehicle's entry and exit information is normal.

[0179] S1203: Determine that the vehicle's entry and exit information is abnormal.

[0180] If the motion trajectory disappears and the occupancy status of the platform parking space remains occupied, it indicates that the original occupancy status of the platform parking space is occupied. After the vehicle exits the platform, the parking space may be abnormally blocked by cargo, etc. The image acquisition device can capture images of the parking space. In this way, the electronic device can detect and analyze the captured images and determine that the occupancy status of the parking space is blocked. The vehicle's entry and exit information can be determined as abnormal vehicle exit.

[0181] As can be seen, in this embodiment, when the target state is to exit the platform, the electronic device can determine the vehicle's departure information based on the motion trajectory and the occupancy status of the platform's parking spaces. This optimizes the problem of occupancy status errors caused by occupancy and enhances adaptability.

[0182] As an implementation method of the embodiment of the present invention, Figure 13 As shown, the above method may further include:

[0183] S1301, perform door detection on the vehicle image to determine the switch status of the vehicle compartment door. If the target state is entering the platform and the switch status is closed, execute step S1302; if the target state is exiting the platform and the switch status is open, execute step S1303.

[0184] When a vehicle enters or leaves a platform, in order to facilitate the loading and unloading operations of platform workers and ensure the safety of the vehicle, electronic equipment can use a deep learning model to detect the doors of the vehicle compartments in the vehicle image, thereby determining the switch status of the vehicle compartment doors, which can include open and closed states.

[0185] S1302, output the abnormal information of the first door.

[0186] The target state is "entering the platform," indicating that the vehicle is about to park at a parking space on the platform and requires personnel to perform loading and unloading operations. At this point, the vehicle's compartment door should be open to allow personnel to load and unload cargo. If the door is closed, the electronic device can output a first compartment door abnormality message, indicating that the compartment door is closed and personnel cannot perform loading and unloading operations.

[0187] S1303, output the abnormal information of the second door.

[0188] The target state is exiting the platform, indicating that the vehicle has completed loading and unloading operations and is about to leave the platform. At this time, the vehicle's compartment doors should be closed to ensure safe and smooth transportation of goods. If the doors are open, the electronic device can output a second compartment door abnormality message. This second compartment door abnormality message indicates that the compartment door is open, which may cause property damage during the vehicle's operation and pose a safety hazard.

[0189] As can be seen, in this embodiment, the electronic device can determine whether the open and close status of the vehicle's compartment door meets the alarm conditions when the vehicle enters or exits the platform. If so, the electronic device can also issue an abnormality message. This avoids affecting the loading and unloading operations of the operators, and also reduces property losses and safety hazards.

[0190] As an implementation of an embodiment of the present invention, the above method may further include at least one of the following steps:

[0191] Step A: If the target state is entering the platform, pedestrian detection is performed on the vehicle image. If the reversing area of ​​the vehicle image includes a pedestrian, a first alarm message is output.

[0192] To reduce safety risks when vehicles enter and exit the platform, electronic devices can use deep learning models to detect pedestrians in vehicle images. The reversing area is the ROI for pedestrian detection by the electronic device, which can specifically be a polygonal area formed by a preset trigger line and lane lines. In this way, if the electronic device detects the presence of a pedestrian in the reversing area, it can output a first alarm message. The first alarm message can be used to remind the driver of the vehicle that the reversing behavior poses a safety risk and that the driver should pay attention to pedestrians when reversing to avoid personal injury. The electronic device can also upload the first alarm message to the vehicle management system. If a pedestrian is detected in the reversing area, the first alarm message is output to remind the driver to pay attention to pedestrians when reversing, thereby reducing the safety risk of vehicles driving in the platform area.

[0193] Step B: If the target state is entering the platform, the vehicle position detection is performed on the vehicle image, and if the vehicle position of the vehicle does not conform to the parking position, a second alarm message is output.

[0194] To improve parking compliance, electronic devices can use deep learning models to detect vehicle positions in vehicle images. This detection includes both the vehicle's position and the markings on the parking spaces on the platform. Standard parking behavior requires the vehicle to park within a parking space and avoid crossing the markings. If the electronic device detects a vehicle parked across a space or crossing the markings, it can output a secondary alarm message to remind the driver to move to the correct parking location. This secondary alarm message, reminding the driver to park correctly, improves parking compliance and, in turn, enhances work efficiency.

[0195] Step C: If the target state is to leave the platform and the loading rate of the carriage is less than a preset threshold, output a third alarm message.

[0196] Electronic devices can use deep learning models to detect the loading rate of vehicle compartments in real time. Different loading rate thresholds can be preset based on actual loading requirements. For example, they can be 80%, 85%, etc. When a vehicle leaves the platform, the electronic devices can compare the loading rate of the compartment with the preset loading rate threshold. If the electronic devices detect that the loading rate of the compartment is less than the preset threshold, they can output a third alarm message. This is used to remind platform operators that the loading rate is not full. By outputting this third alarm message to remind operators that the loading rate is not full, the operators' loading efficiency and work standardization can be improved.

[0197] It can be seen that in this embodiment, the electronic device can issue different alarm information according to different detection results, thereby reducing the safety risk of vehicle driving and improving the operation efficiency and the standardization of the operation of the operators.

[0198] The following combination Figure 14The method for determining the loading rate of a vehicle provided in the embodiment of the present invention is described with examples. Figure 14 As shown, this may include:

[0199] S1401, collecting images of vehicles entering;

[0200] The collected vehicle entry images are all images taken when the vehicle enters the platform area, and the images may include carriage images, platform images, vehicle images, etc.

[0201] S1402, determining the door open / close status when entering the vehicle;

[0202] When the vehicle enters the platform and the door is in the closed state, the first door abnormality information is output.

[0203] S1403, pedestrian detection and alarm;

[0204] When the vehicle enters the platform and there are pedestrians in the reversing area, a first alarm message is output.

[0205] S1404, parking position detection and alarm;

[0206] When it is detected that the vehicle position of the vehicle does not conform to the parking position, a second alarm message is output.

[0207] S1405, determining vehicle entry information;

[0208] The vehicle entry information is determined based on the movement trajectory and the occupancy status of the parking spaces on the platform.

[0209] S1406, determining the vehicle loading rate;

[0210] Acquire an image of the cargo compartment of a loaded vehicle, use monocular depth estimation, and use a depth estimation model to obtain the distance from the cargo surface of the vehicle to the image acquisition device. Use an integral algorithm to obtain the free volume of the compartment, and then determine the vehicle's loading rate.

[0211] S1407, determine the number of operating personnel;

[0212] A deep learning model is used to identify people in platform images and determine the number of workers included in the platform images.

[0213] S1408, calculating the loading efficiency of the vehicle;

[0214] The vehicle's loading efficiency is calculated based on the carriage's loading rate, operation time, and the number of operators.

[0215] S1409, determining the vehicle's departure information;

[0216] The vehicle's departure information is determined based on the movement trajectory and the occupancy status of the parking spaces on the platform.

[0217] S1410, collecting images of vehicles driving out;

[0218] The collected vehicle exit images are all images taken when the vehicle enters the platform area, and the images may include carriage images, platform images, vehicle images, etc.

[0219] S1411, loading rate judgment and alarm;

[0220] When the vehicle leaves the platform and detects that the loading rate of the carriage is less than the preset threshold, the third alarm information is output.

[0221] S1412, determining the door's open / close status when leaving the vehicle;

[0222] When the vehicle leaves the platform and the door switch state is open, the second door abnormal information is output.

[0223] In the steps of S1401, capturing images of vehicles entering the vehicle; S1410, capturing images of vehicles exiting the vehicle, the electronic device can upload the captured images to the vehicle management system. In S1406, determining the vehicle loading rate; S1407, determining the number of workers; and S1408, calculating the vehicle loading efficiency, the electronic device can upload and update the loading rate, number of workers, and loading efficiency in real time to the vehicle management system.

[0224] It can be seen that in this embodiment, the electronic device can obtain the vehicle's loading rate accurately and at low cost, and has strong generalizability. The electronic device can calculate the vehicle's loading efficiency, realizing the detection of working status and efficiency. The electronic device can determine the vehicle's entry and exit information based on the motion trajectory and the occupancy status of the platform's parking spaces, optimizing the pain points such as occupancy status errors and target trajectory loss caused by occupancy, and vehicle entry and exit in blind spots of the field of view, thereby enhancing adaptability. The electronic device can also reduce safety risks during vehicle driving through pedestrian detection and door detection; improve parking standardization through parking position detection; and improve the loading efficiency and operation standardization of operators through loading rate judgment.

[0225] Corresponding to the above-mentioned method for determining the vehicle loading rate, an embodiment of the present invention further provides a device for determining the vehicle loading rate. The following introduces a device for determining the vehicle loading rate provided by an embodiment of the present invention.

[0226] like Figure 15 As shown, a vehicle loading rate determination device, the device comprising:

[0227] The compartment image acquisition module 1501 is used to acquire the compartment image of the vehicle loaded with goods;

[0228] The vehicle compartment image is an image captured by an image acquisition device from the rear of the vehicle toward the interior of the vehicle compartment.

[0229] The distance calculation module 1502 is used to input the vehicle compartment image into a depth estimation model and obtain the distance from the cargo surface of the vehicle to the image acquisition device through monocular depth estimation.

[0230] The loading rate determination module 1503 is configured to determine the loading rate of the vehicle based on the distance between the cargo surface and the image acquisition device.

[0231] It can be seen that in the solution provided by the embodiment of the present invention, the electronic device can obtain an image of the compartment of a vehicle loaded with cargo, wherein the compartment image is an image captured by the image acquisition device from the rear of the vehicle toward the interior of the compartment; and the compartment image is input into the depth estimation model. Through monocular depth estimation, the electronic device can obtain the distance from the cargo side of the vehicle to the image acquisition device; the electronic device can also determine the loading rate of the vehicle based on the distance from the cargo side to the image acquisition device. The electronic device can perform monocular depth estimation through the depth estimation model to determine the distance from the cargo side of the vehicle to the image acquisition device, and then determine the loading rate of the vehicle based on the distance from the cargo side to the image acquisition device. Since the use of monocular depth estimation does not require the installation of complex depth information acquisition equipment, the loading rate of the vehicle can be obtained at low cost and accurately.

[0232] As an implementation of an embodiment of the present invention, the loading rate determination module 1503 may include:

[0233] The free volume calculation unit is used to obtain the free volume of the carriage by using an integral algorithm based on the distance from the cargo surface to the image acquisition device.

[0234] The loading rate determining unit is used to calculate the loading rate of the carriage according to the volume of the carriage and the free volume.

[0235] As an implementation manner of the embodiment of the present invention, the above-mentioned device may further include:

[0236] The platform image acquisition module is used to acquire the platform image of the platform where the vehicle is parked.

[0237] The number determination module is used to perform personnel recognition on the platform image and determine the number of operating personnel included in the platform image.

[0238] The loading operation efficiency calculation module is used to calculate the loading operation efficiency of the vehicle according to the loading rate, operation time and quantity of the vehicle compartment.

[0239] As an implementation manner of the embodiment of the present invention, the above-mentioned device may further include:

[0240] A vehicle image acquisition module, configured to acquire a plurality of vehicle images of the vehicle in a target state;

[0241] The target state includes entering the platform and / or exiting the platform.

[0242] The motion trajectory determination module is used to perform vehicle detection on the vehicle image and determine the motion trajectory of the vehicle.

[0243] An entry and exit information determination module is used to determine the entry and exit information of the vehicle based on the movement trajectory and the occupancy status of the parking spaces on the platform.

[0244] As an implementation manner of an embodiment of the present invention, the above-mentioned motion trajectory determination module includes:

[0245] A vehicle identification unit, configured to perform vehicle identification on the vehicle image to obtain a plurality of vehicle target identification results;

[0246] Wherein, the vehicle target recognition result includes the vehicle position.

[0247] The license plate recognition unit is used to perform license plate recognition on the vehicle image to obtain multiple license plate target recognition results.

[0248] The association and matching unit is used to associate and match the vehicle target recognition result with the license plate target recognition result to obtain a detection result queue.

[0249] The motion trajectory determination unit is used to determine the motion trajectory of the vehicle based on the acquisition time and vehicle position corresponding to the matching vehicle target recognition result and the license plate target recognition result in the detection result queue.

[0250] As an implementation method of an embodiment of the present invention, the target state is entering the platform; the entry and exit information determination module may include:

[0251] The first judgment module is used to obtain the occupancy status of the parking spaces on the platform and to judge whether the motion track disappears.

[0252] The first vehicle entry module is configured to determine that the vehicle entry and exit information is normal entry if the motion trajectory disappears and the occupancy status of the parking space on the platform changes from idle to occupied.

[0253] The second vehicle entry module is configured to determine that the vehicle entry and exit information is an abnormal entry if the motion trajectory disappears and the occupied status of the parking space on the platform remains idle.

[0254] The third vehicle entry module is configured to determine that the vehicle entry and exit information is a normal entry if the motion trajectory disappears and the occupancy status of the parking space on the platform remains occupied and the original occupancy status of the parking space is a blocked state.

[0255] The fourth vehicle entry module is configured to determine that the vehicle entry and exit information is abnormal entry if the motion trajectory disappears and the occupancy status of the parking space on the platform remains occupied and the original occupancy status of the parking space is an existing vehicle status.

[0256] As an implementation method of an embodiment of the present invention, the target state is exiting the platform; the entry and exit information determination module may include:

[0257] The second judgment module is used to obtain the occupancy status of the parking spaces on the platform and to judge whether the motion track disappears.

[0258] The first vehicle exit module is configured to determine that the vehicle's entry and exit information is normal if the motion trajectory disappears and the occupancy status of the parking space on the platform changes from occupied to free.

[0259] The second vehicle exit module is configured to determine that the vehicle's entry and exit information is abnormal if the motion trajectory disappears and the occupancy status of the parking space on the platform remains occupied.

[0260] As an implementation manner of the embodiment of the present invention, the above-mentioned device may further include:

[0261] The vehicle door detection module is used to perform vehicle door detection on the vehicle image and determine the open / close state of the vehicle compartment door.

[0262] The first door abnormality information output module is used to output the first door abnormality information if the target state is entering the platform and the switch state is closed.

[0263] The second door abnormality information output module is used to output the second door abnormality information if the target state is to leave the platform and the switch state is the open state.

[0264] As an implementation manner of the embodiment of the present invention, the above-mentioned device may further include:

[0265] A first alarm information output module is configured to, if the target state is entering the platform, perform pedestrian detection on the vehicle image, and output a first alarm message if a pedestrian is included in the reversing area of ​​the vehicle image; or

[0266] A second alarm information output module is configured to, if the target state is entering the platform, perform vehicle position detection on the vehicle image, and output a second alarm message if the vehicle position does not conform to the parking position; or

[0267] The third alarm information output module is configured to output a third alarm information if the target state is to leave the platform and the loading rate of the carriage is less than a preset threshold.

[0268] An embodiment of the present invention further provides a vehicle management system, which is described below.

[0269] like Figure 16 As shown, a vehicle management system includes an image acquisition device 1601 and an integrated management platform 1602, wherein:

[0270] The image acquisition device 1601 is used to execute the method steps described in any of the above embodiments.

[0271] The integrated management platform 1602 is used to obtain the platform information reported by the image acquisition device, wherein the platform information includes at least one of image information, vehicle information, parking space occupancy status, and alarm information.

[0272] It can be seen that in the solution provided by the embodiment of the present invention, the electronic device can capture the platform image through the image acquisition device, and then perform monocular depth estimation through the depth estimation model to determine the distance from the cargo surface of the vehicle to the image acquisition device, and then determine the vehicle's loading rate based on the distance from the cargo surface to the image acquisition device. Since the use of monocular depth estimation does not require the installation of complex depth information acquisition equipment, the vehicle's loading rate can be obtained at low cost and accurately.

[0273] The vehicle management system also enables digital vehicle management for the logistics park. By combining vehicle entry information with a digital map of the park, the system enables comprehensive management of the park, platforms, vehicles, cargo, and personnel. This comprehensive solution and comprehensive functionality enable cost reduction and efficiency improvement.

[0274] Image acquisition devices are used within vehicle management systems to capture image information. They can be one or more of a smart camera, a parking space detection camera, or a network camera, without limitation. These devices can capture platform information, including images of loaded vehicle compartments, images of docked platforms, multiple images of vehicles in their intended positions, and other platform-related information, such as vehicles, parking spaces, and alarms. The image acquisition devices can also report this platform information to the integrated management platform.

[0275] The integrated management platform and image acquisition equipment can be connected via network cables or optical fibers. The integrated management platform can obtain platform information reported by the image acquisition equipment. Platform information can include image information, such as images of vehicles, platform workers, and parking spaces. Platform information can also include the occupancy status of each parking space, the movement trajectory of vehicles, and alarm messages when abnormalities occur on the platform. The integrated management platform can integrate the data included in the platform information to complete digital vehicle management for the logistics park.

[0276] As an implementation method of the embodiment of the present invention, Figure 17 As shown, the above system may further include a platform management platform 1701;

[0277] The platform management platform 1701 can be used to obtain the platform information stored in the integrated management platform and the parking space reservation information sent by the reservation system 1702, and complete the parking space reservation based on the platform information and the reservation information.

[0278] The reservation system 1702 may be a third-party parking reservation system. The reservation system 1702 is connected to the platform management platform 1701 via a network and is used to send parking reservation information to the platform management platform 1701 .

[0279] The above system may further include a display screen 1703 for displaying the platform status on the electronic map based on the parking space reservation information, vehicle information and the occupancy status of the parking space.

[0280] The above system may further include a fill light 1704 for providing fill light to the image acquisition device 1601 based on the ambient light intensity.

[0281] The above system can also include a switch 1705, which is used to receive the platform information collected by the image acquisition device 1601 and send it to the integrated management platform 1602; it is also used to receive the fill light information sent by the integrated management platform 1602 and send it to the fill light 1704; it is also used to receive the display information sent by the platform management platform 1701 and send it to the controller 1706 of the display screen 1703.

[0282] The above system may further include a controller 1706 for controlling the display screen 1703 to display information.

[0283] The platform management platform and the integrated management platform can be connected through the network. The platform management platform can obtain the platform information stored in the integrated management platform, and can also obtain the parking space reservation information sent by the reservation system. The platform information includes the occupancy status of the parking space. After the platform management platform receives the parking space reservation information sent by the reservation system, it can select a parking space with an occupancy status of free from the platform information and update the status of the parking space to the reserved status. In this way, before the reserved vehicle stops at the parking space, other vehicles cannot stop at the parking space. After the reserved vehicle completes the loading and unloading operation and drives out of the platform, the platform management platform can update the occupancy status of the parking space to free. In this way, the reserved vehicle can quickly complete the loading and unloading operation, realize the digital management of parking spaces, and improve work efficiency.

[0284] As an implementation method of the embodiment of the present invention, Figure 17 As shown, the above system may further include a display screen 1703;

[0285] The platform management platform 1701 may also be used to control the display screen to display the platform status on the electronic map based on the parking space reservation information, the vehicle information, and the occupancy status of the parking space.

[0286] The display screen is connected to the platform management platform via a controller and switches. The screen can display an electronic map of the logistics park, which includes multiple platforms. The platform management platform can also control the display screen to display platform status on the electronic map based on parking space reservation information, vehicle information, and parking space occupancy status. For example, depending on the current status of the platform, the platform status can be displayed as "free," "reserved," or "in use." If the platform status is "reserved" or "in use," the platform management platform can also control the display screen to display the corresponding license plate information based on the parking space information and license plate information reported by the image acquisition device. This allows for the visualization of multiple data on the electronic map.

[0287] If the platform management platform detects an alarm, it can control the display screen to display the alarm message and, if necessary, sound an alarm. For example, if a vehicle entering platform 235 detects a pedestrian in the vehicle's reversing area, the platform management platform will output the first alarm message. After detecting the first alarm message, the platform management platform can control the display screen to display "Pedestrians in the reversing area of ​​platform 235, please pay attention" on the electronic map.

[0288] By controlling the information displayed on the display screen, the platform management platform can uniformly display all platform information in the logistics park, and can visualize multiple data on the electronic map, realizing visual management of vehicles, improving operational efficiency and reducing labor costs.

[0289] As an implementation method of the embodiment of the present invention, Figure 17 As shown, the above system may further include a fill light 1704;

[0290] The integrated management platform 1602 may also be used to control the fill light to provide fill light for the image acquisition device based on the ambient light intensity.

[0291] The fill light is connected to the integrated management platform via a switch. The integrated management platform can obtain real-time ambient light intensity from sensors on the fill light or from other external devices. This allows the fill light to be controlled based on ambient light intensity, providing supplemental lighting for the image acquisition device according to pre-set fill light rules. For example, at night or on rainy days, when lighting conditions are poor, the integrated management platform can control the fill light based on ambient light intensity to provide supplemental lighting for the image acquisition device, allowing the device to capture images normally.

[0292] The embodiment of the present invention further provides an electronic device, for example, an image acquisition device, such as Figure 18 As shown, it includes a processor 1801, a communication interface 1802, a memory 1803 and a communication bus 1804, wherein the processor 1801, the communication interface 1802, and the memory 1803 communicate with each other through the communication bus 1804.

[0293] Memory 1803, used for storing computer programs;

[0294] The processor 1801 is configured to implement the method steps described in any of the above embodiments when executing the program stored in the memory 1803 .

[0295] The communication bus mentioned in the electronic device mentioned above may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0296] The communication interface is used for communication between the above electronic device and other devices.

[0297] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0298] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0299] In another embodiment of the present invention, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the method described in any of the above embodiments are implemented.

[0300] In another embodiment of the present invention, a computer program product including instructions is provided, which, when executed on a computer, enables the computer to execute the method steps described in any one of the above embodiments.

[0301] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0302] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0303] Each embodiment in this specification is described in a related manner. Similar portions between embodiments can be referenced to each other. Each embodiment focuses on the differences between other embodiments. In particular, the device, electronic device, computer-readable storage medium, and computer program product embodiments are generally similar to the method embodiments, so their descriptions are relatively simple. For related portions, reference can be made to the descriptions of the method embodiments.

[0304] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention are included in the scope of protection of the present invention.

Claims

1. A method for determining a vehicle loading rate, characterized in that: The method comprises: Acquire a compartment image of a vehicle loaded with cargo, wherein the compartment image is an image captured by an image acquisition device from the rear of the vehicle toward the interior of the compartment; Inputting the vehicle compartment image into a depth estimation model, and obtaining the distance from the cargo surface of the vehicle to the image acquisition device through monocular depth estimation; determining a loading rate of the vehicle based on a distance from the cargo surface to the image acquisition device; Acquiring a plurality of vehicle images of the vehicle in a target state, wherein the target state includes entering a platform and / or exiting a platform; Performing vehicle detection on the vehicle image to determine a movement trajectory of the vehicle; Determining the vehicle's entry and exit information based on the movement trajectory and the occupancy status of the parking spaces on the platform; The target state is entering a platform; and the step of determining the vehicle's entry and exit information based on the motion trajectory and the occupancy status of the parking spaces on the platform includes: Obtain the occupancy status of the parking space at the platform and determine whether the motion trajectory disappears; if the motion trajectory disappears and the occupancy status of the parking space at the platform changes from idle to occupied, determine that the vehicle's entry and exit information is normal entry; if the motion trajectory disappears and the occupancy status of the parking space at the platform remains idle, determine that the vehicle's entry and exit information is abnormal entry; if the motion trajectory disappears and the occupancy status of the parking space at the platform remains occupied, when the original occupancy status of the parking space is a blocked state, determine that the vehicle's entry and exit information is normal entry; if the motion trajectory disappears and the occupancy status of the parking space at the platform remains occupied, when the original occupancy status of the parking space is an existing vehicle state, determine that the vehicle's entry and exit information is abnormal entry; The target state is exiting the platform; and the step of determining the vehicle's entry and exit information based on the motion trajectory and the occupancy status of the parking spaces on the platform includes: Obtain the occupancy status of the parking space on the platform and determine whether the motion trajectory disappears; if the motion trajectory disappears and the occupancy status of the parking space on the platform changes from occupied to free, determine that the vehicle's entry and exit information is normal exit; if the motion trajectory disappears and the occupancy status of the parking space on the platform remains occupied, determine that the vehicle's entry and exit information is abnormal exit.

2. The method according to claim 1, characterized in that The step of determining the loading rate of the vehicle based on the distance from the cargo surface to the image acquisition device includes: Based on the distance between the cargo surface and the image acquisition device, an integral algorithm is used to obtain the free volume of the carriage; The loading rate of the carriage is calculated based on the volume of the carriage and the free volume.

3. The method according to claim 1 or 2, characterized in that The method further comprises: Acquire a platform image of the platform where the vehicle is parked; Performing personnel recognition on the platform image to determine the number of operating personnel included in the platform image; The loading operation efficiency of the vehicle is calculated based on the loading rate, operation time and quantity of the vehicle compartment.

4. The method according to claim 1, wherein The step of performing vehicle detection on the vehicle image to determine the movement trajectory of the vehicle includes: Performing vehicle recognition on the vehicle image to obtain a plurality of vehicle target recognition results, wherein the vehicle target recognition results include vehicle positions; Performing license plate recognition on the vehicle image to obtain multiple license plate target recognition results; Associating and matching the vehicle target recognition result with the license plate target recognition result to obtain a detection result queue; The movement trajectory of the vehicle is determined based on the acquisition time and vehicle position corresponding to the matching vehicle target recognition result and the license plate target recognition result in the detection result queue.

5. The method according to claim 1, wherein The method further comprises: Performing door detection on the vehicle image to determine the open / close state of the vehicle compartment door; If the target state is entering the platform and the switch state is closed, output first door abnormality information; If the target state is to leave the platform and the switch state is open, the second door abnormality information is output.

6. The method according to claim 1, characterized in that The method further comprises at least one of the following steps: If the target state is entering the platform, performing pedestrian detection on the vehicle image, and outputting a first alarm message if a pedestrian is included in the reversing area of ​​the vehicle image; or, If the target state is entering the platform, performing vehicle position detection on the vehicle image, and outputting a second alarm message if the vehicle position does not conform to the parking position; or If the target state is to leave the platform and the loading rate of the carriage is less than a preset threshold, a third alarm message is output.

7. A device for determining a vehicle loading rate, characterized in that: The device comprises: A compartment image acquisition module, configured to acquire an image of a compartment of a vehicle loaded with cargo, wherein the compartment image is an image acquired by an image acquisition device from the rear of the vehicle toward the interior of the compartment; a distance calculation module, configured to input the vehicle compartment image into a depth estimation model and obtain the distance from the cargo side of the vehicle to the image acquisition device through monocular depth estimation; a loading rate determining module, configured to determine a loading rate of the vehicle based on a distance from the cargo surface to the image acquisition device; a vehicle image acquisition module, configured to acquire a plurality of vehicle images of the vehicle in a target state, wherein the target state includes entering the platform and / or exiting the platform; a motion trajectory determination module, configured to perform vehicle detection on the vehicle image and determine the motion trajectory of the vehicle; an entry and exit information determination module, configured to determine the entry and exit information of the vehicle based on the motion trajectory and the occupancy status of the parking spaces on the platform; Wherein, the target state is entering the platform; the entry and exit information determination module includes: a first judgment module, configured to obtain the occupancy status of the parking spaces on the platform and determine whether the motion trajectory disappears; a first vehicle entry module, configured to determine that the vehicle entry and exit information is a normal entry if the motion trajectory disappears and the occupancy status of the parking space on the platform changes from idle to occupied; a second vehicle entry module, configured to determine that the vehicle entry and exit information is an abnormal entry if the motion trajectory disappears and the occupancy status of the parking space on the platform remains vacant; a third vehicle entry module, configured to determine that the vehicle entry and exit information is a normal entry if the motion trajectory disappears and the occupancy state of the parking space on the platform remains occupied and the original occupancy state of the parking space is a blocked state; a fourth vehicle entry module, configured to determine that the vehicle entry and exit information is an abnormal vehicle entry if the motion trajectory disappears and the occupancy status of the parking space on the platform remains occupied and the original occupancy status of the parking space is an existing vehicle status; The target state is to leave the platform; the entry and exit information determination module includes: a second judgment module, configured to obtain the occupancy status of the parking spaces on the platform and to judge whether the motion trajectory disappears; a first vehicle exit module, configured to determine that the vehicle's entry and exit information is a normal vehicle exit if the motion trajectory disappears and the occupancy status of the parking space on the platform changes from occupied to free; The second vehicle exit module is configured to determine that the vehicle's entry and exit information is abnormal if the motion trajectory disappears and the occupancy status of the parking space on the platform remains occupied.

8. A vehicle management system, characterized in that: The system includes an image acquisition device and an integrated management platform, wherein: The image acquisition device is used to perform the method steps according to any one of claims 1 to 6; The integrated management platform is used to obtain the platform information reported by the image acquisition device, wherein the platform information includes at least one of image information, vehicle information, parking space occupancy status, and alarm information.

9. The system according to claim 8, characterized in that The system also includes a platform management platform; The platform management platform is used to obtain the platform information stored in the integrated management platform and the parking space reservation information sent by the reservation system, and complete the parking space reservation based on the platform information and the reservation information.

10. The system according to claim 9, characterized in that The system also includes a display screen; The platform management platform is further used to control the display screen to display the platform status on the electronic map based on the parking space reservation information, the vehicle information and the occupancy status of the parking space.

11. The system according to any one of claims 8 to 10, characterized in that: The system also includes a fill light; The integrated management platform is further used to control the fill light to provide fill light for the image acquisition device based on the ambient light intensity.

12. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory for storing computer programs; A processor, configured to implement the method steps described in any one of claims 1 to 6 when executing a program stored in a memory.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps of any one of claims 1 to 6 are implemented.

Images