Vehicle load mass identification methods, equipment, vehicles, storage media, and chips

Abstract

This invention provides a vehicle loading quality identification method, device, vehicle, storage medium, and chip. The vehicle loading quality identification method includes: acquiring video; processing the video data, and annotating the processed video with bounding boxes to obtain training data, the bounding boxes including a first bounding box indicating loading completion and a second bounding box indicating loading incomplete; training a basic detection model using the training data to obtain a loading quality identification artificial intelligence model; and analyzing the video data using the loading quality identification artificial intelligence model to determine whether the number of bounding boxes is greater than a first threshold of the video frame rate within a first preset time period, and whether the proportion of the first bounding box in the total number of bounding boxes is greater than a second threshold. If so, output first loading information indicating loading completion; otherwise, output second loading information indicating loading incomplete.

Description

Technical Field

[0001] This invention relates to the field of vehicle inspection technology, and more specifically, to a method for identifying vehicle load quality, a data processing device, a vehicle, a computer-readable storage medium, and a chip. Background Technology

[0002] Intelligent recognition of loading quality in unmanned mining trucks is an important means to improve the operational efficiency of unmanned mining operations and accelerate the automation and unmanned construction of mining. After the excavator completes the loading of minerals onto the unmanned mining truck, the truck must receive a loading completion instruction before it can begin transportation.

[0003] In related technologies, the determination of whether a load is qualified or not is mainly done manually. This will affect the achievement of the goal of fully automated mining and transportation. Furthermore, due to the subjective differences in human judgment, problems such as inconsistent judgment standards are prone to occur, resulting in low efficiency and inaccurate identification of loading quality.

[0004] How to achieve rapid and accurate identification of loading quality is an urgent problem to be solved. Summary of the Invention

[0005] In order to solve or improve at least one of the above-mentioned technical problems, one object of the present invention is to provide a method for identifying vehicle load quality.

[0006] Another object of the present invention is to provide a data processing device.

[0007] Another object of the present invention is to provide a vehicle.

[0008] Another object of the present invention is to provide a computer-readable storage medium.

[0009] Another object of the present invention is to provide a chip.

[0010] To achieve the above objectives, the first aspect of the present invention provides a method for identifying vehicle loading quality, comprising: acquiring a video; processing the video data, and annotating the processed video with bounding boxes to obtain training data, wherein the bounding boxes include a first bounding box indicating that loading is complete and a second bounding box indicating that loading is incomplete; training a basic detection model using the training data to obtain an artificial intelligence model for loading quality identification; and analyzing the video data using the artificial intelligence model for loading quality identification to determine whether the number of bounding boxes is greater than a first threshold of the video frame rate within a first preset time period, and whether the proportion of the first bounding box in the total number of bounding boxes is greater than a second threshold; if so, outputting first loading information indicating that loading is complete, otherwise outputting second loading information indicating that loading is incomplete.

[0011] According to the technical solution of the vehicle loading quality identification method provided by the present invention, a vehicle loading quality identification method is provided that can quickly and accurately identify the loading quality of vehicles. The identification process does not require manual intervention and achieves a high degree of intelligence. Furthermore, the advantages of the method of the present invention are as follows: First, the model can be automatically iteratively trained based on labeled loading data information, learning the characteristic information of the loading container (such as the truck bed) and the materials, without requiring manual intervention in the training process, thus obtaining a highly accurate and robust identification model (loading quality identification artificial intelligence model); Second, the judgment of loading quality is based on AI (artificial intelligence) technology and video frame rate, resulting in high accuracy and stable judgment speed; Third, the method of the present invention can be deployed on data processing equipment, which can be located in the driver's cab or operator's cab of the vehicle, eliminating the need for fixed-position measuring equipment at the loading and unloading site, thus reducing costs and facilitating mobility.

[0012] Specifically, the steps of the vehicle load mass identification method include:

[0013] The first step is to acquire video. Optionally, acquire the loading operation video stream; determine the loading video data based on the loading operation video stream. A video stream refers to the transmission of video data. Compared to the loading operation video stream, the loading video data has reduced pixel values ​​or lower resolution. The loading video data is mainly used for training the model in subsequent steps; the loading operation video stream is mainly used for data analysis using the trained model in subsequent steps. Optionally, acquire video through an information acquisition module. Optionally, the information acquisition module is a loading data acquisition module. Optionally, the information acquisition module includes a camera, which is located in the driver's cab or operator's cab of the vehicle. The vehicle here can be an excavator or a loading vehicle, etc. Ensure that the camera can completely observe the process of the excavator loading minerals into the loading vehicle's bucket. Optionally, acquire video with a duration exceeding a third threshold. The third threshold is 3 hours. Limiting the duration of the acquired video helps ensure the accuracy of the trained model and the final data analysis results.

[0014] The second step involves data processing of the video. The processed video is then labeled with bounding boxes to obtain training data. These bounding boxes include a first bounding box indicating successful loading and a second bounding box indicating incomplete loading. Optionally, the loaded video data can be further processed by frame extraction, data cleaning, and target labeling to obtain training data, which is used to train the model in subsequent steps.

[0015] Specifically, the loading video data is extracted at a rate of 1 frame / second to obtain an image dataset. The image dataset is then cleaned, retaining only data where the loading is complete and material information is clearly visible, as training data. Data cleaning is the final step in identifying and correcting identifiable errors in the data files, including checking data consistency and handling invalid and missing values. The cleaned image dataset is then labeled with bounding boxes. These bounding boxes are also called target boxes. There are two types of bounding boxes: one representing the first bounding box ("full") indicating complete loading, and the other representing the second bounding box ("loading") indicating incomplete loading.

[0016] The third step involves training the basic detection model using training data to obtain a loading quality recognition AI model. Optionally, the basic detection model is the YOLO detection model. The YOLO detection model is used for object detection, detecting objects in an image and identifying their location within the image. Optionally, the training data is split into a training set, a validation set, and a test set in a 7:1:2 ratio. The training set is used to estimate the model; the validation set is used to determine parameters that determine the network structure or control the model's complexity; and the test set is used to evaluate the performance of the ultimately selected optimal model. Optionally, the data input size is 640×640 (pixels or resolution). Optionally, the learning rate is 0.001. In machine learning and statistics, the learning rate is a tuning parameter in optimization algorithms. This parameter determines the step size in each iteration, causing the loss function to converge to its minimum. The learning rate controls how much the model weights are updated based on the estimation error each time. Optionally, SGD (Stochastic Gradient Descent) is used as the training optimizer. The batch size is 16, and 300 iterations are performed. Through the above methods, the neural network model can fully learn information such as the shape, size, and color of the loading vehicle's cargo bed, as well as the shape, color, and area of ​​the materials.

[0017] The trained model is a loading quality recognition AI model. This model is deployed on a data processing device. Optionally, the data processing device is an edge AI (artificial intelligence) device that performs recognition on the loading operation video stream. To conserve computing power, the loading operation video stream is recognized every other frame; for example, after recognizing the first frame, the second frame is ignored, the third frame is recognized, and so on, to identify the loading quality and output the target detection result.

[0018] This invention is based on a deep learning target detection network. The model can be automatically iteratively trained based on labeled loading data to learn the feature information of loading containers and materials. No manual intervention is required in the training process to obtain a highly accurate and robust recognition model (loading quality recognition artificial intelligence model).

[0019] The fourth step involves using an AI-powered quality recognition model to analyze the video data. This determines whether the number of bounding boxes exceeds a first threshold of the video's frame rate within a first preset time period, and whether the percentage of bounding boxes within the first set of bounding boxes exceeds a second threshold. If so, first loading information indicating successful loading is output; otherwise, second loading information indicating incomplete loading is output. Determining successful loading requires meeting two conditions: First, the number of bounding boxes exceeds the first threshold of the video's frame rate within the first preset time period. Second, the percentage of bounding boxes within the first set of bounding boxes exceeds the second threshold within the first preset time period. Optionally, the first preset time period is 3 seconds; the first threshold is 80%; and the second threshold is 90%.

[0020] Optionally, the target detection results can be analyzed temporally using a loading quality recognition AI model. Temporal analysis refers to analyzing data results over a time period. If the number of bounding boxes output within 3 seconds is greater than 80% of the video's frame rate, and the proportion of bounding boxes (first bounding boxes) of the full category exceeds 90%, then loading is considered complete, and first loading information indicating completion is output; otherwise, loading is considered incomplete, and second loading information indicating incomplete loading is output.

[0021] This invention determines the loading quality based on AI (artificial intelligence) technology and video frame rate, achieving high accuracy and stable speed.

[0022] The technical solution defined in this invention provides a method for identifying vehicle loading quality, which can quickly and accurately identify the loading quality of vehicles without human intervention, achieving a high degree of intelligence. Furthermore, the advantages of this method are as follows: First, the model can automatically iteratively train based on labeled loading data, learning the characteristic information of the loading container (e.g., the truck bed) and materials, without requiring manual intervention in the training process, thus obtaining a highly accurate and robust identification model (an AI model for loading quality identification); Second, the determination of loading quality is based on AI (artificial intelligence) technology and video frame rate, resulting in high accuracy and stable judgment speed; Third, the method of this invention can be deployed on data processing equipment, which can be located in the driver's cab or operator's cab of the vehicle, eliminating the need for fixed measuring equipment at the loading and unloading site, thus reducing costs and facilitating mobility.

[0023] In addition, the technical solution provided by the present invention may also have the following additional technical features:

[0024] In the above technical solution, acquiring video includes: acquiring a loading operation video stream; determining loading video data based on the loading operation video stream; processing the video data, and obtaining training data by target annotation of the processed video using bounding boxes, wherein the bounding boxes include a first bounding box indicating loading completion and a second bounding box indicating loading incompleteness; specifically: processing the loading video data, and obtaining training data by target annotation of the processed loading video data using bounding boxes, wherein the bounding boxes include a first bounding box indicating loading completion and a second bounding box indicating loading incompleteness; analyzing the video data using a loading quality recognition artificial intelligence model to determine whether the number of bounding boxes within a first preset time period is greater than a first threshold of the video frame rate, and whether the proportion of the first bounding box in the bounding boxes is greater than a second threshold; if so, outputting first loading information indicating loading completion, otherwise outputting second loading information indicating loading incompleteness; specifically: analyzing the loading operation video stream using a loading quality recognition artificial intelligence model to determine whether the number of bounding boxes within a first preset time period is greater than a first threshold of the loading operation video stream frame rate, and whether the proportion of the first bounding box in the bounding boxes is greater than a second threshold; if so, outputting first loading information indicating loading completion, otherwise outputting second loading information indicating loading incompleteness.

[0025] In this technical solution, the specific steps for acquiring video include:

[0026] Acquire the loading operation video stream. A video stream refers to the transmission of video data. The loading operation video stream is primarily used to determine the loading video data and for data analysis in subsequent steps using the trained model. Optionally, the loading operation video stream is acquired through an information acquisition module. Optionally, the information acquisition module is a loading data acquisition module. Optionally, the information acquisition module includes a camera, which is located in the driver's cab or operator's cab of the vehicle. The vehicle here can be an excavator or a loading vehicle, etc. Ensure that the camera can fully observe the process of the excavator loading minerals into the loading vehicle's bucket. Optionally, acquire video with a duration exceeding a third threshold. The third threshold is 3 hours. Limiting the duration of the acquired video helps ensure the accuracy of the trained model and the final data analysis results.

[0027] Loading video data is determined based on the loading operation video stream. Compared to the loading operation video stream, the loading video data has reduced pixel values ​​or lower resolution. The loading video data is primarily used for training the model in subsequent steps. Optionally, the information acquisition module determines the loading video data based on the loading operation video stream.

[0028] Further, the video undergoes data processing and target annotation to obtain training data, specifically as follows:

[0029] The loading video data is processed, and the processed loading video data is labeled with bounding boxes to obtain training data. The bounding boxes include a first bounding box indicating that loading is complete and a second bounding box indicating that loading is incomplete. Optionally, the loading video data is further processed by frame extraction, data cleaning, and target labeling to obtain training data, which is used to train the model in subsequent steps.

[0030] Specifically, the loading video data is extracted at a rate of 1 frame / second to obtain an image dataset. The image dataset is then cleaned, retaining only data where the loading is complete and material information is clearly visible, as training data. Data cleaning is the final step in identifying and correcting identifiable errors in the data files, including checking data consistency and handling invalid and missing values. The cleaned image dataset is then labeled with bounding boxes. These bounding boxes are also called target boxes. There are two types of bounding boxes: one representing the first bounding box ("full") indicating complete loading, and the other representing the second bounding box ("loading") indicating incomplete loading.

[0031] Furthermore, the process of analyzing video data by loading a quality recognition artificial intelligence model is as follows:

[0032] An AI model for loading quality recognition analyzes the loading operation video stream to determine whether the number of bounding boxes exceeds a first threshold of the loading operation video stream's frame rate within a first preset time period, and whether the percentage of the first bounding boxes within the bounding boxes exceeds a second threshold. If so, it outputs first loading information indicating loading is complete; otherwise, it outputs second loading information indicating loading is incomplete. Determining whether loading is complete requires meeting two conditions: First, the number of bounding boxes exceeds the first threshold of the loading operation video stream's frame rate within the first preset time period. Second, the percentage of the first bounding boxes within the bounding boxes exceeds the second threshold within the first preset time period. Optionally, the first preset time period is 3 seconds; the first threshold is 80%; and the second threshold is 90%.

[0033] Optionally, a loading quality recognition artificial intelligence model can be used to perform time-series analysis on the target detection results. Time-series analysis refers to analyzing the data results within a time period. If the number of bounding boxes output within 3 seconds is greater than 80% of the frame rate of the loading operation video stream, and the proportion of bounding boxes (first bounding boxes) of the full category exceeds 90%, then loading is determined to be complete, and the first loading information indicating that loading is complete is output; otherwise, loading is determined to be incomplete, and the second loading information indicating that loading is incomplete is output.

[0034] In the above technical solution, the loaded video data is processed, and the processed loaded video data is labeled with bounding boxes to obtain training data. The bounding boxes include a first bounding box indicating that loading is complete and a second bounding box indicating that loading is incomplete. The process includes: extracting frames from the loaded video data to obtain an image dataset; cleaning the image dataset; and labeling the cleaned image dataset with bounding boxes to obtain training data. The bounding boxes include a first bounding box indicating that loading is complete and a second bounding box indicating that loading is incomplete.

[0035] In this technical solution, training data is obtained by processing the loaded video data and annotating the targets. The specific steps include:

[0036] Frames are extracted from the loaded video data to obtain an image dataset. Optionally, frames are extracted from the loaded video data at a rate of 1 frame / second. Frame extraction of the loaded video data helps improve the processing speed in subsequent steps.

[0037] Data cleaning is performed on the image dataset. After cleaning, data with complete hopper contents and clearly observable material information is retained as training data. Data cleaning is the final procedure to identify and correct identifiable errors in the data files, including checking data consistency and handling invalid and missing values.

[0038] The cleaned image dataset is used to generate training data through bounding box annotation. The bounding boxes include a first bounding box indicating successful loading and a second bounding box indicating incomplete loading. These bounding boxes can also be called target boxes. There are two types of bounding boxes: the first bounding box, labeled "full," and the second bounding box, labeled "loading," indicating incomplete loading.

[0039] In the above technical solution, if otherwise, output the second loading information indicating that the loading is not completed, specifically: if otherwise, output the second loading information indicating that the loading is not completed, and return to obtain the video.

[0040] In this technical solution, if otherwise, a second loading message indicating that loading is incomplete is output, specifically:

[0041] Otherwise, output a second loading message indicating that loading is incomplete, and return to retrieve the video. If one or both conditions for determining whether loading is complete are not met, output a second loading message indicating that loading is incomplete, and repeat the above steps.

[0042] In the above technical solution, the first preset time is 3s; and / or the first threshold is 80%; and / or the second threshold is 90%.

[0043] In this technical solution, by controlling the first preset time, firstly, the first preset time will not be too short, ensuring the accuracy of the data analysis results; secondly, the first preset time will not be too long, ensuring the efficiency of the data analysis.

[0044] By setting the first threshold to 80%, it is determined whether the number of bounding boxes within the first preset time is greater than 80% of the frame rate of the loading operation video stream, so as to ensure that the loading video data after data processing has been labeled with targets.

[0045] By setting the second threshold to 90%, under the premise of satisfying the first condition (the number of bounding boxes is greater than the first threshold of the frame rate of the loading operation video stream within a first preset time), if the proportion of the number of the first bounding box in the bounding box is greater than 90%, that is, the proportion of the bounding box (first bounding box) of the full category exceeds 90%, then the loading is determined to be completed, and the first loading information indicating that the loading is completed is output; otherwise, the loading is determined to be incomplete, and the second loading information indicating that the loading is incomplete is output.

[0046] In the above technical solution, the training data includes a training set, a validation set, and a test set, with the training set accounting for 70% of the training data; and / or the validation set accounting for 10% of the training data; and / or the test set accounting for 20% of the training data.

[0047] In this technical solution, the training data is split into a training set, a validation set, and a test set in a ratio of 7:1:2. The training set is used to estimate the model; the validation set is used to determine parameters for the network structure or to control the model's complexity; and the test set is used to evaluate the performance of the ultimately selected optimal model. Through this process, the neural network model fully learns information such as the shape, size, and color of the loading truck bed, as well as the shape, color, and area of ​​the materials.

[0048] This invention is based on a deep learning target detection network. The model can be automatically iteratively trained based on labeled loading data to learn the feature information of loading containers and materials. No manual intervention is required in the training process to obtain a highly accurate and robust recognition model (loading quality recognition artificial intelligence model).

[0049] A second aspect of the present invention provides a data processing device, comprising: an information acquisition module for acquiring video; a data processing module for processing the video data, and for annotating the processed video with bounding boxes to obtain training data, the bounding boxes including a first bounding box indicating loading completion and a second bounding box indicating loading incomplete; a model training module for training a basic detection model using the training data to obtain a loading quality recognition artificial intelligence model; and a data analysis module for analyzing the video data using the loading quality recognition artificial intelligence model, determining whether the number of bounding boxes is greater than a first threshold of the video frame rate within a first preset time period, and whether the proportion of the first bounding box in the total number of bounding boxes is greater than a second threshold; if so, outputting first loading information indicating loading completion; otherwise, outputting second loading information indicating loading incomplete.

[0050] According to the technical solution of the data processing device of the present invention, the data processing device includes an information acquisition module, a data processing module, a model training module, and a data analysis module. The information acquisition module is used to acquire video. Optionally, it acquires a loading operation video stream; and determines loading video data based on the loading operation video stream. A video stream refers to the transmission of video data. Compared to the loading operation video stream, the loading video data has reduced pixel values ​​or lower resolution. The loading video data is mainly used for training the model in subsequent steps; the loading operation video stream is mainly used for data analysis using the trained model in subsequent steps. Optionally, the information acquisition module is a loading data acquisition module. Optionally, the information acquisition module includes a camera, which is located in the driver's cab or operator's cab of a vehicle. The vehicle here can be an excavator or a loading vehicle, etc. It ensures that the camera can completely observe the process of the excavator loading minerals into the loading vehicle's bucket. Optionally, it acquires video with a duration exceeding a third threshold. The third threshold is 3 hours. By limiting the duration of the acquired video, it is beneficial to ensure the accuracy of the trained model and the final data analysis results.

[0051] Furthermore, the data processing module is used to process the video data. The processed video is then labeled with bounding boxes to obtain training data. These bounding boxes include a first bounding box indicating successful loading and a second bounding box indicating incomplete loading. Optionally, the loaded video data is further processed by frame extraction, data cleaning, and target labeling to obtain training data, which is used to train the model in subsequent steps.

[0052] Specifically, the loading video data is extracted at a rate of 1 frame / second to obtain an image dataset. The image dataset is then cleaned, retaining only data where the loading is complete and material information is clearly visible, as training data. Data cleaning is the final step in identifying and correcting identifiable errors in the data files, including checking data consistency and handling invalid and missing values. The cleaned image dataset is then labeled with bounding boxes. These bounding boxes are also called target boxes. There are two types of bounding boxes: one representing the first bounding box ("full") indicating complete loading, and the other representing the second bounding box ("loading") indicating incomplete loading.

[0053] Further, the model training module is used to train the basic detection model using training data to obtain a loaded quality recognition AI model. Optionally, the basic detection model is the YOLO detection model. The YOLO detection model is used for object detection, detecting objects in images and identifying their locations within the images. Optionally, the training data is split into a training set, a validation set, and a test set in a 7:1:2 ratio. The training set is used to estimate the model; the validation set is used to determine parameters that determine the network structure or control the model's complexity; and the test set is used to evaluate the performance of the ultimately selected optimal model. Optionally, the data input size is 640×640 (pixels or resolution). Optionally, the learning rate is 0.001. In machine learning and statistics, the learning rate is a tuning parameter in optimization algorithms. This parameter determines the step size in each iteration, causing the loss function to converge to its minimum. The learning rate controls how much the model weights are updated each time based on the estimation error. Optionally, SGD (Stochastic Gradient Descent) is used as the training optimizer. The batch size is 16, and 300 iterations of training are performed. Through the above methods, the neural network model can fully learn information such as the shape, size, and color of the loading vehicle's cargo bed, as well as the shape, color, and area of ​​the materials.

[0054] The trained model is a loading quality recognition AI model. This model is deployed on a data processing device. Optionally, the data processing device is an edge AI (artificial intelligence) device that performs recognition on the loading operation video stream. To conserve computing power, the loading operation video stream is recognized every other frame; for example, after recognizing the first frame, the second frame is ignored, the third frame is recognized, and so on, to identify the loading quality and output the target detection result.

[0055] This invention is based on a deep learning target detection network. The model can be automatically iteratively trained based on labeled loading data to learn the feature information of loading containers and materials. No manual intervention is required in the training process to obtain a highly accurate and robust recognition model (loading quality recognition artificial intelligence model).

[0056] Furthermore, the data analysis module is used to analyze the video data using a loading quality recognition AI model to determine whether the number of bounding boxes exceeds a first threshold of the video's frame rate within a first preset time period, and whether the percentage of the first bounding boxes in the total number of bounding boxes exceeds a second threshold. If so, it outputs first loading information indicating successful loading; otherwise, it outputs second loading information indicating incomplete loading. Determining whether loading is complete requires meeting two conditions: The first condition is that the number of bounding boxes exceeds the first threshold of the video's frame rate within the first preset time period. The second condition is that the percentage of the first bounding boxes in the total number of bounding boxes within the first preset time period exceeds the second threshold. Optionally, the first preset time period is 3 seconds; the first threshold is 80%; and the second threshold is 90%.

[0057] Optionally, the target detection results can be analyzed temporally using a loading quality recognition AI model. Temporal analysis refers to analyzing data results over a time period. If the number of bounding boxes output within 3 seconds is greater than 80% of the video's frame rate, and the proportion of bounding boxes (first bounding boxes) of the full category exceeds 90%, then loading is considered complete, and first loading information indicating completion is output; otherwise, loading is considered incomplete, and second loading information indicating incomplete loading is output.

[0058] This invention determines the loading quality based on AI (artificial intelligence) technology and video frame rate, achieving high accuracy and stable speed.

[0059] In the technical solution defined by this invention, the data processing device is used to execute the vehicle loading quality identification method in any of the above technical solutions, which can quickly and accurately identify the vehicle loading quality. The identification process does not require manual intervention and achieves a high degree of intelligence. In addition, the advantages of the method of this invention are as follows: First, the model can be automatically iteratively trained based on labeled loading data information to learn the feature information of the loading container (such as the truck bed) and the material. No manual intervention is required in the training process to obtain a highly accurate and robust identification model (loading quality identification artificial intelligence model); Second, the judgment of loading quality is based on AI (artificial intelligence) technology and video frame rate, resulting in high accuracy and stable judgment speed; Third, the data processing device can be installed in the driver's cab or operator's cab of the vehicle, eliminating the need for fixed measuring equipment at the loading and unloading site, making it low-cost and easy to move.

[0060] A third aspect of the present invention provides a vehicle, including: a driver's cab; and the data processing device described above, disposed in the driver's cab.

[0061] According to the technical solution of the vehicle of the present invention, the vehicle includes a driver's cab and the data processing equipment described above. The data processing equipment is located in the driver's cab. The data processing equipment is used to execute the vehicle loading quality identification method, which can quickly and accurately identify the vehicle loading quality. The identification process does not require manual intervention, achieving a high degree of intelligence. It eliminates the need for fixed measuring equipment at the loading and unloading site, resulting in low cost and easy mobility.

[0062] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the vehicle loading quality identification method in any of the above-described technical solutions.

[0063] The fifth aspect of the present invention provides a chip, the chip including a processor and a communication interface, the communication interface and the processor being coupled, the processor being used to run a program or instructions to implement the steps of the vehicle loading quality identification method in any of the above technical solutions.

[0064] Additional aspects and advantages of the technical solutions of the present invention will become apparent in the following description or may be learned by practice of the invention. Attached Figure Description

[0065] Figure 1 A first flowchart of a vehicle load mass identification method according to an embodiment of the present invention is shown;

[0066] Figure 2 A second flowchart of a vehicle load mass identification method according to an embodiment of the present invention is shown;

[0067] Figure 3 A third flowchart of a vehicle load mass identification method according to an embodiment of the present invention is shown;

[0068] Figure 4 A schematic diagram of a data processing apparatus according to an embodiment of the present invention is shown;

[0069] Figure 5 A schematic diagram of a vehicle according to an embodiment of the present invention is shown.

[0070] in, Figure 4 and Figure 5 The correspondence between the reference numerals and component names in the attached drawings is as follows:

[0071] 400: Data processing equipment; 410: Information acquisition module; 420: Data processing module; 430: Model training module; 440: Data analysis module; 500: Vehicle; 510: Driver's cab. Detailed Implementation

[0072] To better understand the above-described objectives, features, and advantages of the embodiments of the present invention, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0073] Many specific details are set forth in the following description in order to provide a full understanding of this application. However, embodiments of the invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of this application is not limited to the specific embodiments disclosed below.

[0074] The following reference Figures 1 to 5 The present invention describes a vehicle loading quality identification method, a data processing device 400, a vehicle 500, a computer-readable storage medium, and a chip, provided by some embodiments thereof.

[0075] In one embodiment of the invention, such as Figure 1 As shown, the steps of the vehicle loading weight identification method include:

[0076] S102, Acquire video. Optionally, acquire the loading operation video stream; determine the loading video data based on the loading operation video stream. The video stream refers to the transmission of video data. Compared to the loading operation video stream, the loading video data has reduced pixel values ​​or lower resolution. The loading video data is mainly used for training the model in subsequent steps; the loading operation video stream is mainly used for data analysis using the trained model in subsequent steps. Optionally, acquire video through an information acquisition module. Optionally, the information acquisition module is a loading data acquisition module. Optionally, the information acquisition module includes a camera, which is located in the driver's cab or operator's cab of the vehicle. The vehicle here can be an excavator or a loading vehicle, etc. Ensure that the camera can completely observe the process of the excavator loading minerals into the loading vehicle's bucket. Optionally, acquire video with a duration exceeding a third threshold. The third threshold is 3 hours. Limiting the duration of the acquired video helps ensure the accuracy of the trained model and the final data analysis results.

[0077] S104, perform data processing on the video. The processed video is then labeled with bounding boxes to obtain training data. The bounding boxes include a first bounding box indicating successful loading and a second bounding box indicating incomplete loading. Optionally, the loaded video data is further processed by frame extraction, data cleaning, and target labeling to obtain training data, which is used to train the model in subsequent steps.

[0078] Specifically, the loading video data is extracted at a rate of 1 frame / second to obtain an image dataset. The image dataset is then cleaned, retaining only data where the loading is complete and material information is clearly visible, as training data. Data cleaning is the final step in identifying and correcting identifiable errors in the data files, including checking data consistency and handling invalid and missing values. The cleaned image dataset is then labeled with bounding boxes. These bounding boxes are also called target boxes. There are two types of bounding boxes: one representing the first bounding box ("full") indicating complete loading, and the other representing the second bounding box ("loading") indicating incomplete loading.

[0079] S106, Train the basic detection model using training data to obtain a loading quality recognition AI model. Optionally, the basic detection model is a YOLO detection model. The YOLO detection model is used for object detection, detecting objects in an image and identifying their location within the image. Optionally, Split the training data into a training set, validation set, and test set in a 7:1:2 ratio. The training set is used to estimate the model; the validation set is used to determine parameters for network structure or controlling model complexity; and the test set is used to evaluate the performance of the ultimately selected optimal model. Optionally, the data input size is 640×640 (pixels or resolution). Optionally, the learning rate is 0.001. In machine learning and statistics, the learning rate is a tuning parameter in optimization algorithms. This parameter determines the step size in each iteration, causing the loss function to converge to its minimum. The learning rate controls how much the model weights are updated based on the estimation error each time. Optionally, SGD (Stochastic Gradient Descent) is used as the training optimizer. The batch size is 16, and 300 iterations of training are performed. Through the above methods, the neural network model can fully learn information such as the shape, size, and color of the loading vehicle's cargo bed, as well as the shape, color, and area of ​​the materials.

[0080] The trained model is a loading quality recognition AI model. This model is deployed on a data processing device. Optionally, the data processing device is an edge AI (artificial intelligence) device that performs recognition on the loading operation video stream. To conserve computing power, the loading operation video stream is recognized every other frame; for example, after recognizing the first frame, the second frame is ignored, the third frame is recognized, and so on, to identify the loading quality and output the target detection result.

[0081] This invention is based on a deep learning target detection network. The model can be automatically iteratively trained based on labeled loading data to learn the feature information of loading containers and materials. No manual intervention is required in the training process to obtain a highly accurate and robust recognition model (loading quality recognition artificial intelligence model).

[0082] S108: The video data is analyzed using a loading quality recognition AI model to determine whether the number of bounding boxes exceeds a first threshold of the video's frame rate within a first preset time period, and whether the proportion of the first bounding boxes among all bounding boxes exceeds a second threshold. If so, first loading information indicating loading is complete is output; otherwise, second loading information indicating loading is incomplete is output. Determining whether loading is complete requires meeting two conditions: The first condition is that the number of bounding boxes exceeds the first threshold of the video's frame rate within the first preset time period. The second condition is that the proportion of the first bounding boxes among all bounding boxes within the first preset time period exceeds the second threshold. The purpose of setting the first condition is to determine whether the loaded video data after data processing has been target-labeled. The purpose of setting the second condition is to determine whether the first bounding boxes occupy the vast majority of the target-labeled loaded video data. Setting these two conditions helps improve the accuracy of the data analysis results.

[0083] Optionally, the first preset time is 3 seconds; the first threshold is 80%; and the second threshold is 90%.

[0084] Optionally, the target detection results can be analyzed temporally using a loading quality recognition AI model. Temporal analysis refers to analyzing data results over a time period. If the number of bounding boxes output within 3 seconds is greater than 80% of the video's frame rate, and the proportion of bounding boxes (first bounding boxes) of the full category exceeds 90%, then loading is considered complete, and first loading information indicating completion is output; otherwise, loading is considered incomplete, and second loading information indicating incomplete loading is output.

[0085] This invention determines the loading quality based on AI (artificial intelligence) technology and video frame rate, achieving high accuracy and stable speed.

[0086] The technical solution defined in this invention provides a method for identifying vehicle loading quality, which can quickly and accurately identify the loading quality of vehicles without human intervention, achieving a high degree of intelligence. Furthermore, the advantages of this method are as follows: First, the model can automatically iteratively train based on labeled loading data, learning the characteristic information of the loading container (e.g., the truck bed) and materials, without requiring manual intervention in the training process, thus obtaining a highly accurate and robust identification model (an AI model for loading quality identification); Second, the determination of loading quality is based on AI (artificial intelligence) technology and video frame rate, resulting in high accuracy and stable judgment speed; Third, the method of this invention can be deployed on data processing equipment, which can be located in the driver's cab or operator's cab of the vehicle, eliminating the need for fixed measuring equipment at the loading and unloading site, thus reducing costs and facilitating mobility.

[0087] In one embodiment of the invention, such as Figure 2 As shown, the steps of the vehicle loading weight identification method include:

[0088] S202, Acquire the loading operation video stream. The video stream refers to the transmission of video data. The loading operation video stream is mainly used to determine the loading video data and for data analysis in subsequent steps using the trained model. Optionally, the loading operation video stream is acquired through an information acquisition module. Optionally, the information acquisition module is a loading data acquisition module. Optionally, the information acquisition module includes a camera, which is located in the driver's cab or operator's cab of the vehicle. The vehicle here can be an excavator or a loading vehicle, etc. It is ensured that the camera can completely observe the process of the excavator loading minerals into the loading vehicle's bucket. Optionally, video with a duration exceeding a third threshold is acquired. The third threshold is 3 hours. Limiting the duration of the acquired video helps ensure the accuracy of the trained model and the final data analysis results.

[0089] S204, Determine loading video data based on the loading operation video stream. The loading video data has reduced pixel values ​​or lower resolution compared to the loading operation video stream. The loading video data is primarily used for training the model in subsequent steps. Optionally, the information acquisition module determines the loading video data based on the loading operation video stream.

[0090] S206, perform data processing on the loading video data. The processed loading video data is then labeled with bounding boxes to obtain training data. The bounding boxes include a first bounding box indicating loading completion and a second bounding box indicating loading incompleteness. Optionally, the loading video data may be processed by frame extraction, data cleaning, and target labeling to obtain training data, which is used to train the model in subsequent steps.

[0091] Specifically, the loading video data is extracted at a rate of 1 frame / second to obtain an image dataset. The image dataset is then cleaned, retaining only data where the loading is complete and material information is clearly visible, as training data. Data cleaning is the final step in identifying and correcting identifiable errors in the data files, including checking data consistency and handling invalid and missing values. The cleaned image dataset is then labeled with bounding boxes. These bounding boxes are also called target boxes. There are two types of bounding boxes: one representing the first bounding box ("full") indicating complete loading, and the other representing the second bounding box ("loading") indicating incomplete loading.

[0092] S208, Train the basic detection model using training data to obtain a loading quality recognition AI model. Optionally, the basic detection model is a YOLO detection model. The YOLO detection model is used for object detection, detecting objects in an image and identifying their location within the image. Optionally, Split the training data into a training set, validation set, and test set in a 7:1:2 ratio. The training set is used to estimate the model; the validation set is used to determine parameters for network structure or controlling model complexity; the test set is used to evaluate the performance of the ultimately selected optimal model. Optionally, the data input size is 640×640 (pixels or resolution). Optionally, the learning rate is 0.001. In machine learning and statistics, the learning rate is a tuning parameter in optimization algorithms. This parameter determines the step size in each iteration, causing the loss function to converge to its minimum. The learning rate controls how much the model weights are updated based on the estimation error each time. Optionally, SGD (Stochastic Gradient Descent) is used as the training optimizer. The batch size is 16, and 300 iterations of training are performed. Through the above methods, the neural network model can fully learn information such as the shape, size, and color of the loading vehicle's cargo bed, as well as the shape, color, and area of ​​the materials.

[0093] The trained model is a loading quality recognition AI model. This model is deployed on a data processing device. Optionally, the data processing device is an edge AI (artificial intelligence) device that performs recognition on the loading operation video stream. To conserve computing power, the loading operation video stream is recognized every other frame; for example, after recognizing the first frame, the second frame is ignored, the third frame is recognized, and so on, to identify the loading quality and output the target detection result.

[0094] This invention is based on a deep learning target detection network. The model can be automatically iteratively trained based on labeled loading data to learn the feature information of loading containers and materials. No manual intervention is required in the training process to obtain a highly accurate and robust recognition model (loading quality recognition artificial intelligence model).

[0095] S210: The loading operation video stream is analyzed using a loading quality recognition AI model to determine whether the number of bounding boxes exceeds a first threshold of the loading operation video stream's frame rate within a first preset time period, and whether the percentage of the first bounding boxes within the bounding boxes exceeds a second threshold. If so, first loading information indicating loading is complete is output; otherwise, second loading information indicating loading is incomplete is output. Determining whether loading is complete requires meeting two conditions: The first condition is that the number of bounding boxes exceeds a first threshold of the loading operation video stream's frame rate within a first preset time period. The second condition is that the percentage of the first bounding boxes within the bounding boxes within a first preset time period exceeds a second threshold. Optionally, the first preset time period is 3 seconds; the first threshold is 80%; and the second threshold is 90%.

[0096] Optionally, a loading quality recognition artificial intelligence model can be used to perform time-series analysis on the target detection results. Time-series analysis refers to analyzing the data results within a time period. If the number of bounding boxes output within 3 seconds is greater than 80% of the frame rate of the loading operation video stream, and the proportion of bounding boxes (first bounding boxes) of the full category exceeds 90%, then loading is determined to be complete, and the first loading information indicating that loading is complete is output; otherwise, loading is determined to be incomplete, and the second loading information indicating that loading is incomplete is output.

[0097] This invention determines the loading quality based on AI (artificial intelligence) technology and video frame rate, achieving high accuracy and stable speed.

[0098] In one embodiment of the invention, such as Figure 3 As shown, the steps of the vehicle loading weight identification method include:

[0099] S302, Acquire the loading operation video stream. The video stream refers to the transmission of video data. The loading operation video stream is mainly used to determine the loading video data and for data analysis in subsequent steps using the trained model. Optionally, the loading operation video stream is acquired through an information acquisition module. Optionally, the information acquisition module is a loading data acquisition module. Optionally, the information acquisition module includes a camera, which is located in the driver's cab or operator's cab of the vehicle. The vehicle here can be an excavator or a loading vehicle, etc. It is ensured that the camera can completely observe the process of the excavator loading minerals into the loading vehicle's bucket. Optionally, video with a duration exceeding a third threshold is acquired. The third threshold is 3 hours. Limiting the duration of the acquired video helps ensure the accuracy of the trained model and the final data analysis results.

[0100] S304, Determine loading video data based on the loading operation video stream. The loading video data has reduced pixel values ​​or lower resolution compared to the loading operation video stream. The loading video data is primarily used for training the model in subsequent steps. Optionally, the information acquisition module determines the loading video data based on the loading operation video stream.

[0101] S306, Frame extraction is performed on the loaded video data to obtain an image dataset. Optionally, the loaded video data is extracted at a rate of 1 frame / s to obtain the image dataset. Frame extraction of the loaded video data helps to improve the data processing speed in subsequent steps.

[0102] S308, perform data cleaning on the image dataset. After cleaning, retain data where the hopper is intact and material information is clearly visible as training data. Data cleaning is the final procedure to identify and correct identifiable errors in the data file, including checking data consistency and handling invalid and missing values.

[0103] S310: After data cleaning, the image dataset is labeled with bounding boxes to obtain training data. The bounding boxes include a first bounding box indicating that loading is complete and a second bounding box indicating that loading is incomplete. The bounding boxes can also be called target boxes. There are two types of bounding boxes: the first bounding box, labeled "full", indicating that loading is complete; and the second bounding box, labeled "loading", indicating that loading is incomplete.

[0104] S312, train the basic detection model using training data to obtain a loading quality recognition AI model. Optionally, the basic detection model is a YOLO detection model. The YOLO detection model is used for object detection, detecting objects in an image and identifying their location within the image. Optionally, split the training data into a training set, a validation set, and a test set in a 7:1:2 ratio. The training set is used to estimate the model; the validation set is used to determine parameters that determine the network structure or control the model's complexity; and the test set is used to evaluate the performance of the ultimately selected optimal model. Optionally, the data input size is 640×640 (pixels or resolution). Optionally, the learning rate is 0.001. In machine learning and statistics, the learning rate is a tuning parameter in optimization algorithms. This parameter determines the step size in each iteration, causing the loss function to converge to its minimum. The learning rate controls how much the model weights are updated based on the estimation error each time. Optionally, SGD (Stochastic Gradient Descent) is used as the training optimizer. The batch size is 16, and 300 iterations of training are performed. Through the above methods, the neural network model can fully learn information such as the shape, size, and color of the loading vehicle's cargo bed, as well as the shape, color, and area of ​​the materials.

[0105] The trained model is a loading quality recognition AI model. This model is deployed on a data processing device. Optionally, the data processing device is an edge AI (artificial intelligence) device that performs recognition on the loading operation video stream. To conserve computing power, the loading operation video stream is recognized every other frame; for example, after recognizing the first frame, the second frame is ignored, the third frame is recognized, and so on, to identify the loading quality and output the target detection result.

[0106] This invention is based on a deep learning target detection network. The model can be automatically iteratively trained based on labeled loading data to learn the feature information of loading containers and materials. No manual intervention is required in the training process to obtain a highly accurate and robust recognition model (loading quality recognition artificial intelligence model).

[0107] S314, the loading operation video stream is analyzed using a loading quality recognition AI model to determine whether the number of bounding boxes exceeds a first threshold of the loading operation video stream's frame rate within a first preset time period, and whether the percentage of the first bounding boxes within the bounding boxes exceeds a second threshold. If so, first loading information indicating loading is complete is output; otherwise, second loading information indicating loading is incomplete is output. Determining whether loading is complete requires meeting two conditions: The first condition is that the number of bounding boxes exceeds a first threshold of the loading operation video stream's frame rate within a first preset time period. The second condition is that the percentage of the first bounding boxes within the bounding boxes within a first preset time period exceeds a second threshold. Optionally, the first preset time period is 3 seconds; the first threshold is 80%; and the second threshold is 90%.

[0108] Optionally, a loading quality recognition artificial intelligence model can be used to perform time-series analysis on the target detection results. Time-series analysis refers to analyzing the data results within a time period. If the number of bounding boxes output within 3 seconds is greater than 80% of the frame rate of the loading operation video stream, and the proportion of bounding boxes (first bounding boxes) of the full category exceeds 90%, then loading is determined to be complete, and the first loading information indicating that loading is complete is output; otherwise, loading is determined to be incomplete, and the second loading information indicating that loading is incomplete is output.

[0109] This invention determines the loading quality based on AI (artificial intelligence) technology and video frame rate, achieving high accuracy and stable speed.

[0110] In one embodiment of the present invention, if otherwise a second loading message indicating that loading is incomplete is output, specifically:

[0111] Otherwise, output a second loading message indicating that loading is incomplete, and return to retrieve the video. If one or both conditions for determining whether loading is complete are not met, output a second loading message indicating that loading is incomplete, and repeat the above steps.

[0112] In one embodiment of the present invention, the first preset time is 3 seconds. By controlling the first preset time, firstly, the first preset time will not be too short, ensuring the accuracy of the data analysis results; secondly, the first preset time will not be too long, ensuring the efficiency of the data analysis.

[0113] In one embodiment of the present invention, the first threshold is 80%. By setting the first threshold to 80%, it is determined whether the number of bounding boxes within a first preset time period is greater than 80% of the frame rate of the loading operation video stream, so as to ensure that the loaded video data after data processing has been target-labeled.

[0114] In one embodiment of the present invention, the second threshold is 90%. By setting the second threshold to 90%, under the premise of satisfying the first condition (the number of bounding boxes is greater than the first threshold of the frame rate of the loading operation video stream within a first preset time), if the proportion of the number of the first bounding box in the bounding boxes is greater than 90%, that is, the proportion of the bounding box (first bounding box) of the full category exceeds 90%, then the loading is determined to be complete, and the first loading information indicating that the loading is complete is output; otherwise, the loading is determined to be incomplete, and the second loading information indicating that the loading is incomplete is output.

[0115] In one embodiment of the present invention, the training data includes a training set, a validation set, and a test set, wherein the training set accounts for 70% of the training data; and / or the validation set accounts for 10% of the training data; and / or the test set accounts for 20% of the training data.

[0116] The training data was split into a training set, a validation set, and a test set in a 7:1:2 ratio. The training set was used to estimate the model; the validation set was used to determine parameters for network structure or controlling model complexity; and the test set was used to evaluate the performance of the ultimately selected optimal model. This approach allowed the neural network model to fully learn information such as the shape, size, and color of the loading truck bed, as well as the shape, color, and area of ​​the materials.

[0117] This invention is based on a deep learning target detection network. The model can be automatically iteratively trained based on labeled loading data to learn the feature information of loading containers and materials. No manual intervention is required in the training process to obtain a highly accurate and robust recognition model (loading quality recognition artificial intelligence model).

[0118] In one embodiment of the invention, such as Figure 4 As shown, the data processing device 400 includes an information acquisition module 410, a data processing module 420, a model training module 430, and a data analysis module 440. The information acquisition module 410 is used to acquire video. Optionally, it acquires a loading operation video stream; and determines loading video data based on the loading operation video stream. A video stream refers to the transmission of video data. Compared to the loading operation video stream, the loading video data has reduced pixel values ​​or lower resolution. The loading video data is mainly used for model training in subsequent steps; the loading operation video stream is mainly used for data analysis using the trained model in subsequent steps. Optionally, the information acquisition module 410 is a loading data acquisition module. Optionally, the information acquisition module 410 includes a camera, which is located in the cab 510 or operator's room of the vehicle 500. The vehicle 500 here can be an excavator or a loading vehicle, etc. It ensures that the camera can completely observe the process of the excavator loading minerals into the loading vehicle's bucket. Optionally, it acquires video with a duration exceeding a third threshold. The third threshold is 3 hours. Limiting the duration of the acquired videos helps ensure the accuracy of the trained model and the final data analysis results.

[0119] Furthermore, the data processing module 420 is used to process the video data. The processed video is labeled with bounding boxes to obtain training data. The bounding boxes include a first bounding box indicating successful loading and a second bounding box indicating incomplete loading. Optionally, the loaded video data is further processed by frame extraction, data cleaning, and target labeling to obtain training data, which is used to train the model in subsequent steps.

[0120] Specifically, the loading video data is extracted at a rate of 1 frame / second to obtain an image dataset. The image dataset is then cleaned, retaining only data where the loading is complete and material information is clearly visible, as training data. Data cleaning is the final step in identifying and correcting identifiable errors in the data files, including checking data consistency and handling invalid and missing values. The cleaned image dataset is then labeled with bounding boxes. These bounding boxes are also called target boxes. There are two types of bounding boxes: one representing the first bounding box ("full") indicating complete loading, and the other representing the second bounding box ("loading") indicating incomplete loading.

[0121] Further, the model training module 430 is used to train the basic detection model using training data to obtain a loaded quality recognition artificial intelligence model. Optionally, the basic detection model is a YOLO detection model. The YOLO detection model is used for object detection, detecting objects in an image and identifying their location within the image. Optionally, the training data is split into a training set, a validation set, and a test set in a 7:1:2 ratio. The training set is used to estimate the model; the validation set is used to determine parameters that determine the network structure or control the model's complexity; and the test set is used to evaluate the performance of the ultimately selected optimal model. Optionally, the data input size is 640×640 (pixels or resolution). Optionally, the learning rate is 0.001. In machine learning and statistics, the learning rate is a tuning parameter in optimization algorithms. This parameter determines the step size in each iteration, causing the loss function to converge to its minimum. The learning rate controls how much the model weights are updated based on the estimation error each time. Optionally, SGD (Stochastic Gradient Descent) is used as the training optimizer. The batch size is 16, and the training is performed through 300 iterations. This method allows the neural network model to fully learn information such as the shape, size, and color of the truck bed, as well as the shape, color, and area of ​​the materials.

[0122] The trained model is a loading quality recognition AI model. This model is deployed on data processing device 400. Optionally, data processing device 400 is an edge AI (artificial intelligence) device that performs recognition on the loading operation video stream. To conserve computing power, the loading operation video stream is recognized every other frame; for example, after recognizing the first frame, the second frame is ignored, the third frame is recognized, and so on, to identify the loading quality and output the target detection result.

[0123] This invention is based on a deep learning target detection network. The model can be automatically iteratively trained based on labeled loading data to learn the feature information of loading containers and materials. No manual intervention is required in the training process to obtain a highly accurate and robust recognition model (loading quality recognition artificial intelligence model).

[0124] Furthermore, the data analysis module 440 is used to perform data analysis on the video using a loading quality recognition artificial intelligence model to determine whether the number of bounding boxes is greater than a first threshold of the video's frame rate within a first preset time period, and whether the percentage of the first bounding boxes in the total number of bounding boxes is greater than a second threshold. If so, it outputs first loading information indicating that loading is complete; otherwise, it outputs second loading information indicating that loading is incomplete. Determining whether loading is complete requires meeting two conditions: The first condition is that the number of bounding boxes is greater than the first threshold of the video's frame rate within the first preset time period. The second condition is that the percentage of the first bounding boxes in the total number of bounding boxes within the first preset time period is greater than the second threshold. Optionally, the first preset time period is 3 seconds; the first threshold is 80%; and the second threshold is 90%.

[0125] Optionally, the target detection results can be analyzed temporally using a loading quality recognition AI model. Temporal analysis refers to analyzing data results over a time period. If the number of bounding boxes output within 3 seconds is greater than 80% of the video's frame rate, and the proportion of bounding boxes (first bounding boxes) of the full category exceeds 90%, then loading is considered complete, and first loading information indicating completion is output; otherwise, loading is considered incomplete, and second loading information indicating incomplete loading is output.

[0126] This invention determines the loading quality based on AI (artificial intelligence) technology and video frame rate, achieving high accuracy and stable speed.

[0127] In the technical solution defined by this invention, the data processing device 400 is used to execute the vehicle 500 loading quality identification method in any of the above embodiments, which can quickly and accurately identify the loading quality of the vehicle 500. The identification process does not require manual intervention and achieves a high degree of intelligence. In addition, the advantages of the method of this invention are as follows: First, the model can be automatically iteratively trained based on labeled loading data information to learn the feature information of the loading container (such as the truck bed) and the material. No manual intervention is required in the training process to obtain a highly accurate and robust identification model (loading quality identification artificial intelligence model); Second, the judgment of loading quality is based on AI (artificial intelligence) technology and video frame rate, resulting in high accuracy and stable judgment speed; Third, the data processing device 400 can be installed in the driver's cab 510 or the operator's room of the vehicle 500, eliminating the need to set up fixed measuring equipment at the loading and unloading site, which is low-cost and easy to move.

[0128] In one embodiment of the invention, such as Figure 5As shown, vehicle 500 includes a cab 510 and a data processing device 400 as described in the above embodiment. The data processing device 400 is located within the cab 510. The data processing device 400 is used to execute the vehicle 500 loading weight identification method, enabling rapid and accurate identification of the vehicle 500's loading weight. The identification process requires no manual intervention, achieving a high degree of intelligence. It eliminates the need for fixed measuring equipment at the loading and unloading site, resulting in low cost and easy mobility.

[0129] It is worth noting that vehicle 500 here can be excavators, loading vehicles, etc.

[0130] In one embodiment of the present invention, a computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the vehicle load quality identification method in any of the above embodiments.

[0131] In one embodiment of the present invention, the chip includes a processor and a communication interface, the communication interface and the processor being coupled together, the processor being used to run a program or instructions to implement the steps of the vehicle loading quality identification method in any of the above embodiments.

[0132] In this invention, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance; the term "multiple" refers to two or more unless otherwise explicitly defined. The terms "install," "connect," "link," and "fix" should be interpreted broadly. For example, "connect" can be a fixed connection, a detachable connection, or an integral connection; "link" can be a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0133] In the description of this invention, it should be understood that the terms "upper," "lower," "left," "right," "front," "rear," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or unit referred to must have a specific orientation or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0134] In the description of this specification, the terms "one embodiment," "some embodiments," "specific embodiment," etc., refer to a specific feature, structure, material, or characteristic described in connection with that embodiment or example, which is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0135] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for identifying the load mass of a vehicle, characterized in that, include: Get the video; The video is processed, and the processed video is labeled with bounding boxes to obtain training data. The bounding boxes include a first bounding box indicating that loading is complete and a second bounding box indicating that loading is incomplete. The basic detection model is trained using the training data to obtain an artificial intelligence model for load quality recognition; The loading quality recognition artificial intelligence model analyzes the video data to determine whether the number of bounding boxes is greater than a first threshold of the video frame rate within a first preset time, and whether the proportion of the number of the first bounding boxes in the bounding boxes is greater than a second threshold. If so, first loading information indicating that loading is complete is output; otherwise, second loading information indicating that loading is incomplete is output. The acquisition of video includes: Acquire the video stream of the loading operation; Determine loading video data based on the loading operation video stream; The process involves data processing of the video, followed by target annotation of the processed video using bounding boxes to obtain training data. The bounding boxes include a first bounding box indicating successful loading and a second bounding box indicating incomplete loading. Specifically: The loading video data is processed, and the processed loading video data is labeled with targets using the bounding boxes to obtain the training data. The bounding boxes include a first bounding box indicating that loading is complete and a second bounding box indicating that loading is incomplete. The process involves analyzing the video data using the loading quality recognition AI model to determine whether the number of bounding boxes exceeds a first threshold of the video's frame rate within a first preset time period, and whether the proportion of the first bounding boxes in the total number of bounding boxes exceeds a second threshold. If so, first loading information indicating successful loading is output; otherwise, second loading information indicating incomplete loading is output. Specifically: The loading operation video stream is analyzed using the loading quality recognition artificial intelligence model. The loading operation video stream is identified every frame to determine whether the number of bounding boxes is greater than the first threshold of the frame rate of the loading operation video stream within the first preset time, and whether the proportion of the number of the first bounding boxes in the bounding boxes is greater than the second threshold. If so, the first loading information indicating that loading is completed is output; otherwise, the second loading information indicating that loading is not completed is output.

2. The vehicle loading weight identification method according to claim 1, characterized in that, The loading video data is processed, and the processed loading video data is labeled with targets using bounding boxes to obtain the training data. The bounding boxes include a first bounding box indicating loading is complete and a second bounding box indicating loading is incomplete, including: Frames are extracted from the loaded video data to obtain an image dataset; Perform data cleaning on the image dataset; The image dataset after data cleaning is labeled with targets using the bounding boxes to obtain the training data. The bounding boxes include a first bounding box indicating that loading is complete and a second bounding box indicating that loading is incomplete.

3. The vehicle loading weight identification method according to claim 1 or 2, characterized in that, The step of outputting second loading information indicating that loading is incomplete if not otherwise is specifically as follows: Otherwise, output the second loading information indicating that loading is incomplete, and return the video acquisition information.

4. The vehicle loading weight identification method according to claim 1 or 2, characterized in that, The first preset time is 3 seconds; and / or the first threshold is 80%; and / or the second threshold is 90%.

5. The vehicle loading weight identification method according to claim 1 or 2, characterized in that, The training data includes a training set, a validation set, and a test set, wherein the training set accounts for 70% of the training data; and / or the validation set accounts for 10% of the training data; and / or the test set accounts for 20% of the training data.

6. A data processing device, characterized in that, include: The information acquisition module is used to acquire video. The data processing module is used to process the video data and to perform target annotation on the processed video using bounding boxes to obtain training data. The bounding boxes include a first bounding box indicating that loading is complete and a second bounding box indicating that loading is incomplete. The model training module is used to train the basic detection model using the training data to obtain an artificial intelligence model for loading quality recognition. The data analysis module is used to perform data analysis on the video through the loading quality recognition artificial intelligence model to determine whether the number of bounding boxes is greater than a first threshold of the video frame rate within a first preset time, and whether the proportion of the number of the first bounding boxes in the bounding boxes is greater than a second threshold. If so, the module outputs first loading information indicating that loading is complete; otherwise, it outputs second loading information indicating that loading is incomplete.

7. A vehicle, characterized in that, include: Driver's cab; The data processing device as described in claim 6 is located in the driver's cab.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the vehicle load quality identification method as described in any one of claims 1 to 5.

9. A chip, characterized in that, The chip includes a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the steps of the vehicle loading quality identification method as described in any one of claims 1 to 5.

Images