AI-based unattended vehicle weighing method, device and storage medium

Abstract

The present invention relates to the field of intelligent weighing, and in particular to an unattended vehicle weighing method, system, device and storage medium based on AI. In order to solve the problem in the prior art that it is difficult to accurately judge any objects other than the vehicle during the weighing process, and it is impossible to identify the vehicle weighed twice as the same vehicle through vehicle features; the present invention obtains vehicle video image data during the vehicle weighing process through a high-definition camera, and analyzes the vehicle video image data through image recognition technology to determine whether the vehicle is fully weighed, detect whether there are people or other objects on the vehicle, identify vehicle features, identify license plate numbers, and whether the vehicle entering and exiting is the same vehicle, so as to prevent the vehicle from cheating during the weighing process. The present invention uses AI video recognition technology and real-time monitoring of the vehicle weighing process through high-definition cameras, and ultimately realizes the automation and unmanned operation of the vehicle weighing process, reduces manual intervention, and improves weighing efficiency.

Description

Technical Field

[0001] The present invention relates to the field of intelligent weighing, and in particular to an AI-based unattended vehicle weighing method, system, device and storage medium. Background Art

[0002] With the rapid development of the logistics industry, vehicle weighing plays a vital role in cargo transportation. Traditional vehicle weighing methods typically require manual operation, resulting in low efficiency, significant human error, and frequent cheating. To address these issues, the logistics industry has begun exploring the use of unmanned weighing systems to improve weighing efficiency and accuracy.

[0003] Conventional unmanned weighing systems mainly rely on hardware equipment such as infrared detection or radar, weight analysis, card swiping or identification. They use infrared detection or radar positioning to determine whether the vehicle is fully weighed, and then automatically weigh the vehicle through a preset weighing process. However, existing unmanned weighing systems have difficulty accurately determining whether there are people and objects other than the vehicle weighing during the weighing process, and are unable to identify the vehicle being weighed twice as the same vehicle through vehicle characteristics. This may lead to inaccurate weighing data and the possibility of cheating.

[0004] Therefore, there is an urgent need to develop an AI-based unattended vehicle weighing method, system, device and storage medium to overcome the shortcomings of the existing technology. Summary of the Invention

[0005] In order to solve the problem in the prior art that people and objects other than vehicles cannot be identified as the same vehicle through vehicle features during weighing, the present invention provides an AI-based unattended vehicle weighing method, system, device and storage medium.

[0006] In a first aspect, the present application discloses an AI-based unattended vehicle weighing method, comprising the following steps:

[0007] S101 has high-definition cameras installed around the weighing platform to monitor the vehicle weighing process in real time and determine whether the vehicle is fully weighed;

[0008] S102 performs real-time analysis on the vehicle video image data captured by the high-definition camera during the weighing process to determine whether there are people or objects other than the weighing vehicle weighing during the vehicle weighing process, and whether there are people or objects in the weighing area that may affect the vehicle weighing. The target detection algorithm used is Faster R-CNN.

[0009] S103 automatically identifies the vehicle features and license plate numbers when the vehicle enters or leaves the weighing platform, and compares the identification results of the vehicle entering and leaving the platform;

[0010] S104 automatically guides the driver to complete the weighing process through a voice interaction device and a display screen based on the weighing data and the analysis results of the vehicle video image data. After the weighing is completed, an electronic weighing receipt is automatically generated and the data is uploaded to the server;

[0011] During the weighing process, S105 records the vehicle weighing process and automatically captures high-definition images at key points. The video and image data are associated with the vehicle weighing record and saved.

[0012] Furthermore, the step of determining whether the vehicle has been fully weighed specifically includes:

[0013] S201 When the vehicle is weighed at the green traffic light, the high-definition camera starts working and captures the vehicle video image data of the weighing process in real time;

[0014] S202 performs real-time analysis on the vehicle video image data using a target detection algorithm to detect and locate the tire position of the vehicle;

[0015] S203 determines whether the tire is completely located within the weighing platform area based on the detected position information of the vehicle tire;

[0016] S204 When all the detected vehicle tires are completely located within the weighing platform area, the vehicle is fully weighed; when some of the vehicle tires are not completely located within the weighing platform area, the unattended vehicle weighing platform reminds the driver to move the vehicle through a voice interaction device until the vehicle tires are completely located within the weighing platform area.

[0017] Furthermore, the determination of whether there are persons or objects other than the weighing vehicle weighing the vehicle during the vehicle weighing process specifically includes:

[0018] S301: The unattended vehicle weighing system obtains the vehicle video image data monitored by the high-definition camera, performs real-time analysis on the vehicle video image data using a target detection algorithm, and searches the storage unit based on the vehicle license plate number and vehicle characteristics to confirm whether the vehicle is being weighed for the first time in this weighing process; if yes, step S302 is executed; if not, it is the second weighing, and step S304 is executed;

[0019] S302: During the first weighing, the vehicle recognition model is used to determine whether there are any persons or objects that may affect the weighing of the vehicle in the weighing area. If there are any persons or objects that may affect the weighing of the vehicle in the weighing area, the unattended vehicle weighing system issues an alarm through the voice interaction device and refuses to weigh until the relevant personnel have completed the verification.

[0020] S303: After confirming that there are no people or objects in the weighing area that may affect the vehicle weighing, the number and characteristics of people in the vehicle cab are recorded, and the number and characteristics of people are associated with the vehicle license plate number and vehicle characteristics and stored, and the first weighing is completed;

[0021] S304: During the second weighing, the vehicle recognition model is used to determine whether there is a person or object that affects the weighing of the vehicle in the weighing area. If there is a person or object that affects the weighing of the vehicle in the weighing area, the unattended vehicle weighing system issues an alarm through the voice interaction device and refuses to weigh until relevant personnel have completed verification.

[0022] After confirming that there are no people or objects that affect the weighing of the vehicle in the weighing area, S305 compares the number and characteristics of people with those recorded during the first weighing to determine whether they are consistent; if so, calculate the weighing result; if not, the unattended vehicle weighing system issues an alarm and refuses to weigh until the relevant personnel have completed the verification.

[0023] Furthermore, the target detection algorithm used in the vehicle recognition model to determine whether there is a person or object that affects the vehicle weighing in the weighing area is Faster R-CNN, and the loss function of the vehicle recognition model is:

[0024]

[0025] in, represents the overall objective function that needs to be minimized during the training process of the vehicle recognition model, and is used to guide the vehicle recognition model to accurately monitor and identify the target and determine the impact of the target on the weighing process; It is the loss function of the RPN network, used to generate target candidate boxes; is the loss function of the classification network, used to identify the target category; is the bounding box regression loss function, which is used to optimize the position and scale of the target box; To verify the loss function, it is used to confirm whether the detected target will affect the vehicle weighing; is the input image; is the target candidate frame set obtained by the RPN network; is the real label, which is used to represent the target object and its location in the image; is a hyperparameter used to balance the bounding box regression loss function; is a hyperparameter used to balance the validation loss function;

[0026] The calculation formula of the validation loss function is:

[0027]

[0028] in, Represents the target candidate box set Each target candidate box Perform summation; The target candidate box set Middle target candidate boxes; Represents the target candidate box The negative logarithmic probability of containing a vehicle is used to maximize the probability that the target candidate box contains a vehicle; Represents the target candidate box The negative logarithmic probability of not containing obstacles that affect weighing is used to minimize the probability that the candidate box contains obstacles that affect weighing; Indicates the probability that the current target box is a vehicle; Indicates the probability that there is an obstacle in the current target box that will affect weighing;

[0029] After the vehicle recognition model is trained using the loss function, vehicle video image data is input, and the output results of the vehicle recognition model are the position and scale information of the target candidate box, the probability distribution of the category to which each target candidate box belongs, the probability of whether each candidate box will affect the vehicle weighing, and the final target detection and recognition results; when there is a person or object that is located in the weighing area, has a reasonable scale, is classified as a person or affects the weighing, and the probability of affecting the weighing is higher than the set threshold, it is judged that there is a person or object that affects the vehicle weighing in the weighing area; but when the probability of affecting the weighing of all target candidate boxes is lower than the set threshold, and the fact that they are in the weighing area, have a reasonable scale, and are classified as a person or an object that affects the weighing also does not support the conclusion that there is a target that affects the weighing, it is judged that there is no target that affects the vehicle weighing in the weighing area.

[0030] Furthermore, the automatic identification of vehicle features and license plate numbers specifically includes:

[0031] S501 Unattended Vehicle Weighing System acquires vehicle video image data and uses a vehicle feature recognition model to identify vehicle features;

[0032] The vehicle feature recognition model is trained Different CNN models are used to identify vehicle features by fusing the prediction results of the CNN models. Model, predict The probability vector of vehicle features is:

[0033]

[0034] in, Representative The number of categories of vehicle features; Indicates the The M-th vehicle feature predicted by the model belongs to the category probability; Indicates the The model predicts The probability that a vehicle feature does not belong to a known recorded category is used to represent the location vehicle feature; For the The model predicts The probability vector of vehicle features;

[0035] Fusion of each vehicle’s features The prediction results of the models are used to obtain the recognition results of the vehicle features:

[0036]

[0037] in, For the The recognition results of vehicle features; For the The first vehicle feature The weight of the model; Vehicle features, based on the fused recognition results , select the category with the highest probability as the The recognition result of vehicle features is output as Vehicle feature vector of vehicle features;

[0038] S502: The unattended vehicle weighing system obtains vehicle video image data and uses a license plate number recognition model to identify the vehicle's license plate number. The license plate number recognition model first uses an edge detection algorithm to locate the license plate number area and performs character segmentation on the license plate number area. The license plate number recognition model then uses a support vector machine (SVM) algorithm to classify each segmented character to obtain a complete license plate number.

[0039] Wherein, when the SVM algorithm is used to classify each segmented character, the objective function calculation formula of the SVM algorithm is:

[0040]

[0041] in, To minimize the objective function, is the normal vector of the hyperplane, is the bias term of the hyperplane, The degree to which each sample is allowed to deviate from the hyperplane; Used to control the complexity of the SVM algorithm and prevent overfitting, expressed as the bi-norm regularization term of the hyperplane; is the penalty coefficient, which is used to balance the trade-off between classification error and margin maximization; For the The weight of each sample is used to balance the importance of different samples; For the Slack variables for samples; The weight representing the character position; For the The weight of each character position; For the The class labels of the characters; is the number of training samples; is the number of character positions; is the normal vector of the hyperplane The transpose of For the The feature representation obtained after feature extraction of a character image; Used to enhance the classification ability of the SVM algorithm;

[0042] By solving the objective function, the optimal hyperplane parameters are obtained and , so that the SVM algorithm can achieve higher classification performance in the license plate number recognition task, and then arrange each recognized character in order to obtain the vehicle's license plate number;

[0043] At step S503, during weighing, the vehicle characteristics and the license plate number recognition result are searched in the unattended vehicle weighing system to determine whether the vehicle is being weighed for the first time. If so, the vehicle characteristics and the license plate number recognition result are associated and stored. If not, it indicates that the vehicle is being weighed for the second time, and step S504 is executed.

[0044] S504 compares the two weighing results. After the vehicle weighing data meets the preservation conditions, the unattended vehicle weighing system associates the weighing data from the two weighings, calculates the weight of the loaded and unloaded cargo of the vehicle, and the weighing process ends.

[0045] Furthermore, the specific calculation formula for determining whether the vehicle is weighed for the first time is:

[0046]

[0047] in, is the vehicle identity similarity; Used to calculate vehicle feature similarity; Used to calculate the similarity of vehicle license plate numbers; and is the weight coefficient, which is used to balance the contribution of vehicle feature similarity and license plate number similarity to vehicle identity similarity; and They are respectively a vehicle feature vector identified at the first weighing and a vehicle feature vector stored in the unattended vehicle weighing system after the first weighing of one of the vehicles; and The license plate number identified at the first weighing and the license plate number of one of the vehicles stored in the unattended vehicle weighing system after the first weighing are character strings; is the edit distance, which represents the conversion of the license plate number string recognized at the first weighing into the license plate number string stored in the unattended vehicle weighing system after the first weighing of one of the vehicles The minimum number of editing operations required; and the length of the license plate number string stored for the first time and after the first weighing of one of the vehicles stored in the unattended vehicle weighing system;

[0048] Finally, the threshold range is set, when the vehicle identity similarity If the value of is within the set threshold range, the vehicle features and license plate number identified during the first weighing are the same as the vehicle features and license plate number stored in the unattended vehicle weighing system after the first weighing of one of the vehicles, that is, the vehicle is weighed for the second time.

[0049] Furthermore, the conditions for the vehicle weighing data to be saved specifically include:

[0050] The vehicle has completely entered the weighing platform area, and the weighing area contains only the vehicle and the people in the vehicle cab, with no people or objects that may affect the weighing of the vehicle;

[0051] The vehicle features and license plate number recognition are completed; and the person in the vehicle cab is the same as during the first weighing, and no person is identified during the first weighing;

[0052] The reading value of the weighing instrument is stable and continuous for more than 20 times.

[0053] The second aspect is an AI-based unattended vehicle weighing system, which includes a weighing platform with a high-definition camera, a signal light, a voice interaction device, and a display screen installed around the weighing platform. The high-definition camera is used to obtain vehicle video image data; the signal light is used to indicate the use status of the weighing platform; and the voice interaction device and display screen are used to instruct the driver to complete the weighing process.

[0054] An image processing unit is configured to obtain the vehicle video image data captured by the high-definition camera, analyze the vehicle video image data, determine whether the vehicle has been fully weighed, and whether the vehicle characteristics and license plate number are consistent; based on the analysis results of the weighing data and the vehicle video image data, automatically guide the driver to complete the weighing process through a voice interaction device and display screen prompts, and automatically generate electronic document data after weighing is completed, and upload the data to the server;

[0055] The storage unit is used to store videos, images and weighing records during the weighing process.

[0056] The third aspect is the AI-based unmanned vehicle weighing device, including:

[0057] at least one processor;

[0058] at least one memory for storing at least one program;

[0059] When the at least one program is executed by the at least one processor, the at least one processor implements the AI-based unattended vehicle weighing method as described above.

[0060] In a fourth aspect, an AI-based unattended vehicle weighing storage medium stores a computer program thereon, which, when executed by a processor, is used to implement the steps of the above-mentioned AI-based unattended vehicle weighing method.

[0061] Beneficial effects:

[0062] (1) The present invention uses AI video recognition technology and high-definition cameras to monitor the vehicle weighing process in real time, ensuring that the vehicle is parked correctly and avoiding weighing errors. By identifying changes in people and objects on the vehicle, weighing cheating is prevented, ultimately achieving automation and unmanned operation of the vehicle weighing process, reducing manual intervention and improving weighing efficiency.

[0063] (2) After weighing is completed, the system automatically generates an electronic document and uploads it to the server, thereby improving management efficiency. At the same time, the system automatically records the entire weighing process video and captures high-definition images at key nodes, which are associated with key records and saved. When problems arise, a return visit can be made for verification, thereby ensuring the traceability of the weighing process. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 Flow chart of the method of the present invention.

[0065] Figure 2 This is a flow chart of the present invention for determining whether a vehicle is fully weighed.

[0066] Figure 3 This is a flow chart of the present invention for identifying whether there is a person or object in the weighing platform that affects weighing.

[0067] Figure 4 This is a flow chart of the present invention for automatically identifying the type and license plate number of a vehicle.

[0068] Figure 5 The figure is a flow chart of the specific measurement process of unattended vehicle weighing according to the present invention. DETAILED DESCRIPTION

[0069] The present invention will be further described below with reference to the accompanying drawings and examples.

[0070] Example 1

[0071] AI-based unattended vehicle weighing method, as shown in the attached Figure 1-4 Shown are:

[0072] In this embodiment, high-definition cameras are installed around the weighing platform to monitor the vehicle weighing process in real time and determine whether the vehicle is completely weighed.

[0073] Specifically, determine whether the vehicle is fully weighed. Figure 2 Specifically included are:

[0074] The S1001 vehicle was weighed when the traffic light was green. When the traffic light changed from green to red, the unattended vehicle weighing system guided the driver to measure the weight through the voice interaction device and display screen. The high-definition cameras installed around the weighing platform started working and captured the vehicle video image data of the weighing process in real time.

[0075] S1002 performs real-time analysis on the vehicle video image data captured by the high-definition camera, and detects and locates the tire position of the vehicle through image analysis algorithms, such as target detection algorithms.

[0076] S1003 compares the detected vehicle tire position information with the boundary of the weighing platform to determine whether the tire is completely located within the weighing platform area.

[0077] S1004 When all the detected vehicle tires are completely located in the weighing platform area, the vehicle is fully weighed; when a vehicle tire is not completely located in the weighing platform area, the unattended vehicle weighing platform reminds the driver to move the vehicle through the voice interaction device until the vehicle tires are completely located in the weighing platform area.

[0078] Specifically, whether the vehicle is fully weighed is determined by detecting whether the wheels have fully entered the weighing platform. The specific reason is that a common cheating method in weighing is to not fully weigh the front and rear wheels, specifically including not fully weighing the rear wheels or not fully weighing the front wheels to reduce the weight of the vehicle.

[0079] Specifically, when the unattended vehicle weighing platform has reminded the driver to move the vehicle for a period of time but the vehicle has not been fully weighed, the unattended vehicle weighing platform refuses to weigh the vehicle and handles it through relevant personnel.

[0080] In this embodiment, the vehicle video image data captured by the high-definition camera during the weighing process is analyzed in real time to determine whether there are people and objects other than the weighing vehicle weighing during the vehicle weighing process.

[0081] Specifically, when judging whether there are people and objects other than the weighing vehicle, it is necessary to clarify that the people in the cab of the vehicle to be tested are not the people who are weighing. However, during the first and second weighing, the number and characteristics of the people in the cab are detected, and the number and characteristics of the people in the cab detected during the two weighings are compared to confirm that the people are consistent during the two weighings.

[0082] The specific process for determining whether there are people or objects other than the vehicle weighing weighing during the vehicle weighing process is as follows: Figure 3 Shown, including:

[0083] S2001 The unattended vehicle weighing system obtains the vehicle video image data monitored by the high-definition camera, performs real-time analysis on the vehicle video image data through the target detection algorithm, and searches the storage unit by the vehicle license plate number and vehicle characteristics to confirm whether the vehicle is being weighed for the first time in this weighing process; if yes, proceed to step S2002; if not, it is the second weighing, and proceed to step S2004.

[0084] S2002 determines whether there are people or objects that affect the weighing of the vehicle in the weighing area during the first weighing. If there are people or objects that affect the weighing of the vehicle in the weighing area, the unattended vehicle weighing system will issue an alarm through the voice interaction device and refuse to weigh until the relevant personnel have completed the verification.

[0085] Specifically, the vehicle recognition model is used to determine whether there is a person or object in the weighing area that affects the vehicle weighing. The target detection algorithm used is Faster R-CNN, and the loss function of the vehicle recognition model is:

[0086]

[0087] in, represents the overall objective function that needs to be minimized during the training process of the vehicle recognition model, and is used to guide the vehicle recognition model to accurately monitor and identify the target and determine the impact of the target on the weighing process; It is the loss function of the RPN network, used to generate target candidate boxes; is the loss function of the classification network, used to identify the target category; is the bounding box regression loss function, which is used to optimize the position and scale of the target box; To verify the loss function, it is used to confirm whether the detected target will affect the vehicle weighing; is the input image; is the target candidate frame set obtained by the RPN network; is the real label, which is used to represent the target object and its location in the image; is a hyperparameter used to balance the bounding box regression loss function; is a hyperparameter used to balance the validation loss function;

[0088] The introduction of can predict whether the target is a vehicle and whether it is an obstacle that will affect vehicle weighing while training the model, thereby better meeting the needs of unattended vehicle weighing. The calculation formula of the validation loss function is:

[0089]

[0090] in, Represents the target candidate box set Each target candidate box Perform summation; The target candidate box set Middle target candidate boxes; Represents the target candidate box The negative logarithmic probability of containing a vehicle is used to maximize the probability that the target candidate box contains a vehicle; Represents the target candidate box The negative logarithmic probability of not containing obstacles that affect weighing is used to minimize the probability that the candidate box contains obstacles that affect weighing; Indicates the probability that the current target box is a vehicle; Indicates the probability that there is an obstacle in the current target box that will affect weighing. By simultaneously maximizing the vehicle probability and minimizing the obstacle probability, it is possible to more accurately determine whether the detected target will affect the weighing.

[0091] By calculating the loss function of the vehicle recognition model, we can develop an object detection model optimized for unattended vehicle weighing scenarios. This model not only accurately monitors and identifies objects such as people and objects outside the vehicle, but also determines whether they will affect the weighing process based on the object's category and location, providing a more reliable distributed perception foundation for unattended vehicle weighing.

[0092] After the vehicle recognition model training is completed using the loss function, the vehicle video image data is input; after the vehicle video image data is input, the target candidate frame set is obtained through the Faster R-CNN region proposal network. The output results of the vehicle recognition model are the position and scale information of the target candidate frame, the probability distribution of the category to which each target candidate frame belongs, the probability of whether each candidate frame will affect vehicle weighing, and the final target detection and recognition results; when there is a person or object that affects weighing in the weighing area, has a reasonable scale, is classified as a person or an object that affects weighing, and the probability of affecting weighing is higher than the set threshold, it is judged that there is a person or object that affects vehicle weighing in the weighing area; but when the probability of affecting weighing of all target candidate frames is lower than the set threshold, and the judgment factors of being in the weighing area, having a reasonable scale, and being classified as a person or an object that affects weighing also do not support the conclusion that there is a target that affects weighing, it is judged that there is no target that affects vehicle weighing in the weighing area.

[0093] The threshold can be set to different values based on the type and importance of the weighing target. For people and large objects, a higher threshold can be set, as they have a greater impact on the weighing results and higher detection accuracy. For small objects, such as leaves and plastic bags, a lower threshold can be set, as they have a smaller impact on the weighing results.

[0094] S2003, after confirming that there are no people or objects that affect the vehicle weighing in the weighing area, records the number and characteristics of people in the vehicle cab, associates the number and characteristics of people with the vehicle license plate number and vehicle characteristics, and stores them. The first weighing is completed.

[0095] During the second weighing, S2004 uses the vehicle identification model to determine whether there are people or objects in the weighing area that may affect the weighing of the vehicle. If there are people or objects in the weighing area that may affect the weighing of the vehicle, the unattended vehicle weighing system will issue an alarm through the voice interaction device and refuse to weigh until the relevant personnel have completed the verification.

[0096] Specifically, during manual verification, the management personnel can retrieve the vehicle video image data and feature comparison results of the unattended vehicle weighing system during two weighings, manually determine whether the people on the vehicle have changed, and input the verification results into the system. If the verification finds that the people on the vehicle have not changed, the alarm will be canceled and the vehicle will be allowed to leave; if the verification finds that the people on the vehicle have indeed changed, the alarm status will be maintained, and the driver will be required to cooperate with the investigation or complete relevant procedures. The driver can only leave after confirmation.

[0097] After confirming that there are no people or objects that affect the weighing of the vehicle in the weighing area, S2005 compares the people and characteristics with those recorded during the first weighing to determine whether they are consistent; if yes, calculate the weighing result; if not, the unattended vehicle weighing system issues an alarm and refuses to weigh until the relevant personnel have completed the verification.

[0098] Specifically, when a person or object that affects vehicle weighing is detected, it is further identified and recorded, and if necessary, associated with relevant data in the weighing process and stored.

[0099] In this embodiment, when a vehicle enters or exits the weighing platform, the type and license plate number of the vehicle are automatically identified, and the identification results of the entering and exiting vehicles are compared.

[0100] Automatically identify the type and license plate number of the vehicle as shown in the attached Figure 4 Specifically included are:

[0101] The S3001 unattended vehicle weighing system acquires vehicle video image data and uses a vehicle feature recognition model to identify vehicle features.

[0102] Specifically, the vehicle feature recognition model is trained Different CNN models are used to identify vehicle features by fusing the prediction results of the CNN models. Model, predict The probability vector of vehicle features is:

[0103]

[0104] in, Representative The number of categories of vehicle features; Indicates the The M-th vehicle feature predicted by the model belongs to the category probability; Indicates the The model predicts The probability that a vehicle feature does not belong to a known recorded category is used to represent the location vehicle feature; For the The model predicts The probability vector of vehicle features;

[0105] Fusion of each vehicle’s features The prediction results of the models are used to obtain the recognition results of the vehicle features:

[0106]

[0107] in, For the The recognition results of vehicle features; For the The first vehicle feature The weight of the model; Vehicle features, based on the fused recognition results , select the category with the highest probability as the The recognition result of vehicle features is output as Vehicle feature vector of vehicle features;

[0108] The S3002 unattended vehicle weighing system obtains vehicle video image data and uses a license plate number recognition model to identify the vehicle's license plate number. The license plate number recognition model first uses an edge detection algorithm to locate the license plate area and performs character segmentation on the license plate area; then uses the SVM algorithm to classify each segmented character to obtain the complete license plate number.

[0109] Specifically, by combining the recognition results of all vehicle features in S3001 with the license plate number recognition results, complete vehicle information can be obtained.

[0110] Specifically, when the SVM algorithm is used to classify each segmented character, the objective function calculation formula of the SVM algorithm is:

[0111]

[0112] in, To minimize the objective function, is the normal vector of the hyperplane, is the bias term of the hyperplane, Indicates the degree to which each sample is allowed to deviate from the hyperplane; Used to control the complexity of the SVM algorithm and prevent overfitting, expressed as the bi-norm regularization term of the hyperplane; Represents the penalty coefficient, which is used to balance the trade-off between classification error and interval maximization. The larger it is, the more severe the penalty for classification errors will be, and the classification boundary of the SVM algorithm will be closer to the training samples; Representative The weight of each sample is used to balance the importance of different samples; Representative Slack variables for samples; Represents the weight of the character position, which is used to control the importance of character position information in the objective function. The larger it is, the greater the influence of character position information on the classification results; Indicates the The weight of each character position is used to distinguish the importance of characters at different positions; Indicates the The category labels of characters. In license plate number recognition, different characters correspond to different category labels; Represents the number of training samples, that is, the number of character images used to train the SVM algorithm; Represents the number of character positions, that is, the number of characters contained in the image; Represents the normal vector of the hyperplane The transpose of Indicates the The feature representation obtained after feature extraction of the character image is The feature vector of samples; Used to enhance the classification ability of the SVM algorithm;

[0113] By solving the objective function, the optimal hyperplane parameters are obtained and , so that the SVM algorithm can achieve higher classification performance in the license plate number recognition task. At the same time, the character position weight is introduced , which can further improve the accuracy of license plate number recognition and make it better suitable for the scenario of unattended vehicle weighing.

[0114] During weighing, S3003 searches the unattended vehicle weighing system based on the vehicle characteristics and the license plate number recognition result to check whether the vehicle is being weighed for the first time; if so, the vehicle characteristics and the license plate number recognition result are associated and stored; if not, it means that the vehicle is being weighed for the second time, and step S3004 is executed.

[0115] S3004 After the vehicle meets the measurement conditions, the unattended vehicle weighing system associates the weighing data from the two weighings to calculate the weight of the loaded and unloaded cargo of the vehicle.

[0116] Specifically, the specific calculation formula for determining whether the vehicle is weighed for the first time is:

[0117]

[0118] in, Represents the vehicle identity similarity; Represents the calculation of vehicle feature similarity; Represents the number used to calculate the similarity of the vehicle's license plate number; and is the weight coefficient, which is used to balance the contribution of vehicle feature similarity and license plate number similarity to vehicle identity similarity; and The vehicle feature vector identified during the first weighing and the vehicle feature vector of one of the vehicles stored in the storage unit after the first weighing; and The license plate number identified at the first weighing and the license plate number of one of the vehicles stored in the storage unit after the first weighing are character strings; is the edit distance, which is the conversion of the license plate number string recognized at the first weighing into the license plate number string stored in the storage unit after the first weighing of one of the vehicles The minimum number of editing operations required; and The length of the license plate number string stored for the first time and for one of the vehicles stored in the storage unit after the first weighing.

[0119] Finally, the threshold range is set, when the vehicle identity similarity If the value of is within the set threshold range, the vehicle features and license plate number identified during the first weighing are the same as the vehicle features and license plate number stored in the unattended vehicle weighing system after the first weighing of one of the vehicles, that is, the vehicle is weighed for the second time.

[0120] Because the edit distance is the minimum number of edit operations required to transform two strings into one another, the smaller the edit distance, the more similar the two strings are. Calculating similarity using the edit distance effectively balances the effects of edit distance and string length on license plate number similarity, resulting in a more accurate and reasonable similarity value.

[0121] Specifically, when relevant personnel conduct manual verification, they retrieve the image data and recognition results of the two license plate number comparisons from the unattended vehicle weighing system, and manually determine whether the license plate numbers are consistent. If they are consistent, the recognition results are manually modified, the vehicle is allowed to leave and the weighing process continues; if they are inconsistent, the driver is required to cooperate in verifying the vehicle information, and after confirmation, the vehicle information is corrected and the weighing process is repeated.

[0122] Specifically, the conditions for saving vehicle weighing data include:

[0123] 1. Whether the vehicle has completely entered the weighing platform area, that is, whether the vehicle tires have completely entered the weighing platform area; and whether there are only the vehicle to be weighed and the people in the vehicle cab in the weighing platform area, and no other people or objects that may affect the weighing of the vehicle to be tested;

[0124] 2. The vehicle characteristics and license plate number of the vehicle to be weighed are identified. During the second weighing of the vehicle to be weighed, the occupants of the vehicle's cab are the same as during the first weighing. If this is the first weighing of the vehicle to be weighed, no judgment is made.

[0125] 3. The instrument data reading value of the vehicle weight to be tested on the weighing platform is continuous and stable for more than X times.

[0126] Specifically, the vehicle characteristics and license plate number identification of the vehicle to be weighed are completed as follows: after detecting the vehicle characteristics and license plate number of the vehicle to be weighed, it is detected in the storage unit whether the vehicle is being weighed for the first time in this weighing process. If so, the measurement condition of the vehicle characteristics and license plate number identification of the vehicle to be weighed is met, and the vehicle characteristics and license plate number of the weighed vehicle are associated and stored; if not, it is the second weighing in this weighing process, and the vehicle characteristics and license plate number identified this time are compared with the vehicle characteristics and license plate number identified during the first weighing, and it is determined whether the identified vehicle characteristics and license plate numbers during the two weighings are consistent. If they are consistent, this measurement condition is met. If they are inconsistent, the unattended vehicle weighing system will issue an alarm and notify relevant personnel to conduct manual verification.

[0127] Specifically, after the vehicle to be tested meets other measurement conditions and the weighing instrument reading value is continuously stable for more than 20 times, the unattended vehicle weighing system will automatically record and save the vehicle weighing data.

[0128] In this embodiment, based on the analysis results of weighing data and vehicle video image data, the driver is automatically guided to complete the weighing process through voice interaction and display screen prompts. After weighing is completed, an electronic weighing receipt is automatically generated and the data is uploaded to the server.

[0129] Specifically, if the vehicle weighing data is valid and there are no people or objects on board, the unattended vehicle weighing system converts the weighing results into voice information and plays it through the voice interaction device, prompting the driver that the weighing is complete and they can get off the scale. Simultaneously, the weighing results and prompts are displayed on the display screen to guide the driver through the weighing process. If the weighing data is invalid or the person and object do not match between the two weighings, the unattended vehicle weighing system prompts the driver to re-weigh or adjust the person or object on the vehicle through the voice interaction device and display screen to ensure the accuracy of the weighing data.

[0130] Specifically, after weighing is completed, the unattended vehicle weighing system automatically integrates the weighing data, vehicle information, time and other information to generate an electronic weighing document, in which the vehicle information includes vehicle characteristics and license plate number, etc.; at the same time, the generated electronic weighing document is uploaded to the server and associated with the vehicle entry and exit records, vehicle video image data and other information to achieve centralized data management and remote access.

[0131] In this embodiment, during the weighing process, the vehicle weighing process is recorded, and high-definition images are automatically captured at key nodes. The video and images are associated with the vehicle weighing record and saved.

[0132] Specifically, after the vehicle enters the weighing platform, the high-definition camera begins to continuously record the weighing process and record the video data of the entire weighing process; at key nodes in the weighing process, including but not limited to the vehicle being fully weighed, the weighing data being stable, and the weighing being completed, the unattended vehicle weighing system automatically triggers the high-definition camera to shoot high-definition images to capture the vehicle status at critical moments; finally, the recorded video and the captured high-definition images and other data are associated with the vehicle weighing records and saved, and a correspondence between the video, image and weighing data is established. The video and image data can be used to conduct post-audit and analysis of the weighing process to ensure the authenticity and reliability of the weighing data.

[0133] Corresponding to the unattended vehicle weighing method shown in the above embodiment, the AI-based unattended vehicle weighing system includes: a weighing platform, around which are installed a high-definition camera, a signal light, a voice interaction device, and a display screen; the high-definition camera is used to obtain vehicle video image data; the signal light is used to indicate the use status of the weighing platform; the voice interaction device and the display screen are used to instruct the driver to complete the weighing process;

[0134] The image processing unit is used to obtain vehicle video image data captured by a high-definition camera and analyze the vehicle video image data to determine whether the vehicle has been fully weighed, whether the people and objects on the vehicle have changed, and whether the vehicle characteristics and license plate number are consistent. Based on the analysis results of the weighing data and vehicle video image data, the unit automatically guides the driver to complete the weighing process through a voice interaction device and display prompts. After weighing is completed, electronic document data is automatically generated and uploaded to the server.

[0135] The storage unit is used to store videos, images and weighing records during the weighing process.

[0136] This embodiment also provides an AI-based unattended vehicle weighing device, including at least one processor; at least one memory for storing at least one program; when the at least one program is executed by the at least one processor, the at least one processor runs the AI-based unattended vehicle weighing method as described above.

[0137] This embodiment also provides an AI-based unattended vehicle weighing storage medium, which stores a program executable by a processor. When the processor-executable program is executed by a processor, it is used to implement the AI-based unattended vehicle weighing method as described above.

[0138] Example 2

[0139] In this embodiment, the specific measurement process of unattended vehicle weighing is as follows:

[0140] S4001 The vehicle is weighed when the traffic light is green. After the vehicle is weighed, the traffic light turns red and the unattended vehicle weighing system guides the driver to perform the weighing through voice and text.

[0141] After the S4002 vehicle is parked correctly, the vehicle features and license plate number are obtained through AI image and video recognition, and the vehicle weighing task order is obtained based on the license plate number;

[0142] The S4003 system automatically reads the corresponding document information, saves the relevant weighing data, automatically raises the gate, the vehicle unloads, the weighing instrument returns to zero, and the signal light changes from red to green, waiting for the next vehicle to be weighed;

[0143] After loading and unloading goods, the S4004 vehicle goes on the scale and swipes the card for secondary weighing. After the measurement is completed, the unattended vehicle weighing system prompts the driver through voice and text to automatically save the data. The barrier lifts the bar, and after the vehicle is completely unloaded, the barrier falls, the scale instrument returns to zero, and the traffic light turns green, waiting for the next vehicle to be weighed.

[0144] During the entire weighing process of S4005, the system automatically records video and takes photos when the weight is confirmed, and uploads the photos to the system server.

[0145] The above-described embodiments merely represent preferred embodiments of the present invention. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make various modifications, improvements, and substitutions without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. The AI-based unattended vehicle weighing method is characterized by: The following steps are included: S101 has high-definition cameras installed around the weighing platform to monitor the vehicle weighing process in real time and determine whether the vehicle is fully weighed; S102 performs real-time analysis on the vehicle video image data captured by the high-definition camera during the weighing process to determine whether there are people or objects other than the weighing vehicle weighing during the vehicle weighing process. A vehicle recognition model is used to determine whether there are people or objects in the weighing area that may affect the weighing of the vehicle. The target detection algorithm used is Faster R-CNN. S103 automatically identifies the vehicle characteristics and license plate number of the vehicle when the vehicle enters or leaves the weighing platform, and compares the identification results of the vehicle entering and leaving the platform. When weighing, the vehicle characteristics and the license plate number identification results are retrieved in the unattended vehicle weighing system to determine whether the vehicle is the first weighing; S104 automatically guides the driver to complete the weighing process through a voice interaction device and a display screen based on the weighing data and the analysis results of the vehicle video image data. After the weighing is completed, an electronic weighing receipt is automatically generated and the data is uploaded to the server; During the weighing process, S105 records the vehicle weighing process and automatically captures high-definition images at key points. The video and image data are associated with the vehicle weighing record and saved; The specific calculation formula for determining whether the vehicle is weighed for the first time is: ； in, is the vehicle identity similarity; Used to calculate vehicle feature similarity; Used to calculate the similarity of vehicle license plate numbers; and is the weight coefficient, which is used to balance the contribution of vehicle feature similarity and license plate number similarity to vehicle identity similarity; and They are respectively a vehicle feature vector identified at the first weighing and a vehicle feature vector stored in the unattended vehicle weighing system after the first weighing of one of the vehicles; and The license plate number identified at the first weighing and the license plate number of one of the vehicles stored in the unattended vehicle weighing system after the first weighing are character strings; is the edit distance, which represents the conversion of the license plate number string recognized at the first weighing into the license plate number string stored in the unattended vehicle weighing system after the first weighing of one of the vehicles The minimum number of editing operations required; and the length of the license plate number string stored for the first time and after the first weighing of one of the vehicles stored in the unattended vehicle weighing system; Finally, the threshold range is set, when the vehicle identity similarity If the value of is within the set threshold range, the vehicle features and license plate number identified during the first weighing are the same as the vehicle features and license plate number stored in the unattended vehicle weighing system after the first weighing of one of the vehicles, that is, the vehicle is weighed for the second time.

2. The AI-based unattended vehicle weighing method according to claim 1, characterized in that: The target detection algorithm used in the vehicle recognition model to determine whether there is a person or object that affects the vehicle weighing in the weighing area is Faster R-CNN, and the loss function of the vehicle recognition model is: ； in, represents the overall objective function that the vehicle recognition model needs to minimize during the training process, and is used to guide the vehicle recognition model to accurately monitor and identify the target and determine the impact of the target on the weighing process; It is the loss function of the RPN network, used to generate target candidate boxes; is the loss function of the classification network, used to identify the target category; is the bounding box regression loss function, which is used to optimize the position and scale of the target box; To verify the loss function, it is used to confirm whether the detected target will affect the vehicle weighing; is the input image; is the target candidate frame set obtained by the RPN network; is the real label, which is used to represent the target object and its location in the image; is a hyperparameter used to balance the bounding box regression loss function; is a hyperparameter used to balance the validation loss function; The calculation formula of the validation loss function is: ； in, Represents the target candidate box set Each target candidate box Perform summation; The target candidate box set Middle target candidate boxes; Represents the target candidate box The negative logarithmic probability of containing a vehicle is used to maximize the probability that the target candidate box contains a vehicle; Represents the target candidate box The negative logarithmic probability of not containing obstacles that affect weighing is used to minimize the probability that the candidate box contains obstacles that affect weighing; Indicates the probability that the current target box is a vehicle; Indicates the probability that there is an obstacle in the current target box that will affect weighing; After the vehicle recognition model is trained using the loss function, vehicle video image data is input, and the output results of the vehicle recognition model are the position and scale information of the target candidate box, the probability distribution of the category to which each target candidate box belongs, the probability of whether each candidate box will affect the vehicle weighing, and the final target detection and recognition results; when there is a person or object that is located in the weighing area, has a reasonable scale, is classified as a person or affects the weighing, and the probability of affecting the weighing is higher than the set threshold, it is judged that there is a person or object that affects the vehicle weighing in the weighing area; but when the probability of affecting the weighing of all target candidate boxes is lower than the set threshold, and the fact that they are in the weighing area, have a reasonable scale, and are classified as a person or an object that affects the weighing also does not support the conclusion that there is a target that affects the weighing, it is judged that there is no target that affects the vehicle weighing in the weighing area.

3. The AI-based unattended vehicle weighing method according to claim 2, characterized in that: Determining whether the vehicle is fully weighed specifically includes: S201 When the vehicle is weighed at the green traffic light, the high-definition camera starts working and captures the vehicle video image data of the weighing process in real time; S202 performs real-time analysis on the vehicle video image data using a target detection algorithm to detect and locate the tire position of the vehicle; S203 determines whether the tire is completely located within the weighing platform area based on the detected position information of the vehicle tire; S204 When all the detected vehicle tires are completely located within the weighing platform area, the vehicle is fully weighed; when some of the vehicle tires are not completely located within the weighing platform area, the unattended vehicle weighing platform reminds the driver to move the vehicle through a voice interaction device until the vehicle tires are completely located within the weighing platform area.

4. The AI-based unattended vehicle weighing method according to claim 3, characterized in that: The determination of whether there are persons or objects other than the weighing vehicle weighing the vehicle during the vehicle weighing process specifically includes: S301: The unattended vehicle weighing system obtains the vehicle video image data monitored by the high-definition camera, performs real-time analysis on the vehicle video image data using a target detection algorithm, and searches the storage unit based on the vehicle license plate number and vehicle characteristics to confirm whether the vehicle is being weighed for the first time in this weighing process; if yes, step S302 is executed; if not, it is the second weighing, and step S304 is executed; S302: During the first weighing, the vehicle recognition model is used to determine whether there are any persons or objects that may affect the weighing of the vehicle in the weighing area. If there are any persons or objects that may affect the weighing of the vehicle in the weighing area, the unattended vehicle weighing system issues an alarm through the voice interaction device and refuses to weigh until the relevant personnel have completed the verification. S303: After confirming that there are no people or objects in the weighing area that may affect the vehicle weighing, the number and characteristics of people in the vehicle cab are recorded, and the number and characteristics of people are associated with the vehicle license plate number and vehicle characteristics and stored, and the first weighing is completed; S304: During the second weighing, the vehicle recognition model is used to determine whether there is a person or object that affects the weighing of the vehicle in the weighing area. If there is a person or object that affects the weighing of the vehicle in the weighing area, the unattended vehicle weighing system issues an alarm through the voice interaction device and refuses to weigh until relevant personnel have completed verification. After confirming that there are no people or objects that affect the weighing of the vehicle in the weighing area, S305 compares the number and characteristics of people with those recorded during the first weighing to determine whether they are consistent; if so, calculate the weighing result; if not, the unattended vehicle weighing system issues an alarm and refuses to weigh until the relevant personnel have completed the verification.

5. The AI-based unattended vehicle weighing method according to claim 4 is characterized in that: The automatic identification of vehicle features and license plate numbers specifically includes: S501 Unattended Vehicle Weighing System acquires vehicle video image data and uses a vehicle feature recognition model to identify vehicle features; The vehicle feature recognition model is trained Different CNN models are used to identify vehicle features by fusing the prediction results of the CNN models. Model, predict The probability vector of vehicle features is: ； in, Representative The number of categories of vehicle features; Indicates the The M-th vehicle feature predicted by the model belongs to the category probability; Indicates the The model predicts The probability that a vehicle feature does not belong to a known recorded category is used to represent the location vehicle feature; For the The model predicts The probability vector of vehicle features; Fusion of each vehicle’s features The prediction results of the models are used to obtain the recognition results of the vehicle features: ； in, For the The recognition results of vehicle features; For the The first vehicle feature The weight of the model; Vehicle features, based on the fused recognition results , select the category with the highest probability as the The recognition result of vehicle features is output as Vehicle feature vector of vehicle features; S502: The unattended vehicle weighing system obtains vehicle video image data and uses a license plate number recognition model to identify the vehicle's license plate number. The license plate number recognition model first uses an edge detection algorithm to locate the license plate number area and performs character segmentation on the license plate number area. The license plate number recognition model then uses a support vector machine (SVM) algorithm to classify each segmented character to obtain a complete license plate number. Wherein, when the SVM algorithm is used to classify each segmented character, the objective function calculation formula of the SVM algorithm is: ； in, To minimize the objective function, is the normal vector of the hyperplane, is the bias term of the hyperplane, The degree to which each sample is allowed to deviate from the hyperplane; Used to control the complexity of the SVM algorithm and prevent overfitting, expressed as the bi-norm regularization term of the hyperplane; is the penalty coefficient, which is used to balance the trade-off between classification error and margin maximization; For the The weight of each sample is used to balance the importance of different samples; For the Slack variables for samples; The weight representing the character position; For the The weight of each character position; For the The class labels of the characters; is the number of training samples; is the number of character positions; is the normal vector of the hyperplane The transpose of For the The feature representation obtained after feature extraction of a character image; Used to enhance the classification ability of the SVM algorithm; By solving the objective function, the optimal hyperplane parameters are obtained and , so that the SVM algorithm can achieve higher classification performance in the license plate number recognition task, and then arrange each recognized character in order to obtain the vehicle's license plate number; At step S503, during weighing, the vehicle characteristics and the license plate number recognition result are searched in the unattended vehicle weighing system to determine whether the vehicle is being weighed for the first time. If so, the vehicle characteristics and the license plate number recognition result are associated and stored. If not, it indicates that the vehicle is being weighed for the second time, and step S504 is executed. S504 compares the two weighing results. After the vehicle weighing data meets the preservation conditions, the unattended vehicle weighing system associates the weighing data from the two weighing times, calculates the weight of the loaded and unloaded cargo of the vehicle, and the weighing process ends.

6. The AI-based unattended vehicle weighing method according to claim 5, characterized in that: The conditions for saving the vehicle weighing data specifically include: The vehicle has completely entered the weighing platform area, and the weighing area contains only the vehicle and the people in the vehicle cab, with no people or objects that may affect the weighing of the vehicle; The vehicle features and license plate number recognition are completed; and the person in the vehicle cab is the same as during the first weighing, and no person is identified during the first weighing; The reading value of the weighing instrument is stable and continuous for more than 20 times.

7. AI-based unattended vehicle weighing device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the AI-based unattended vehicle weighing method according to any one of claims 1 to 4 are implemented.

8. An AI-based unattended vehicle weighing storage medium storing a processor-executable program, characterized in that: When the processor-executable program is executed by the processor, it is used to implement the AI-based unattended vehicle weighing method described in any one of claims 1-4.

Images