Off-campus vehicle intelligent safety guiding method and system based on target detection technology

Abstract

The invention discloses an off-campus vehicle intelligent safety guiding method and system based on a target detection technology. The system is divided into a user side and an administrator side. Wherein the user side mainly has the functions of providing access registration service for an off-campus vehicle user, providing a driving route according to destination information input by the user and the like. The administrator end is mainly used for providing functions of user information management, destination management, data visualization, real-time monitoring of vehicle speed and license plate and the like for campus administrators. According to the system, the Python language is adopted as a development language in rear-end development, and the vehicle speed and the license plate are detected in combination with a Yolov8 algorithm model. And meanwhile, a ByteTrack algorithm is utilized to realize multi-target tracking. The front end of the user side adopts a WeChat applet as a user interface, and a user can easily access the system through a WeChat applet platform without downloading additional applications to complete required functions. The administrator front end adopts a Vue.js framework to carry out front-end development, so that an administrator can conveniently carry out operations such as user management and destination management.

Description

Technical Field

[0001] The present invention belongs to the technical field of image data processing, provides an off-campus vehicle intelligent safety guidance method and system based on target detection technology, and relates to a YOLOv8 algorithm model for detecting vehicle speed and license plate. Background Art

[0002] The Yolo series of algorithms are single-stage algorithms with simple structures and faster recognition speeds. The Yolo series of algorithms are efficient and accurate, especially the latest Yolov8 algorithm. Yolov8 can detect targets in images in real time and output objects with corresponding bounding boxes. It is widely used to detect objects in scenes, such as vehicles.

[0003] ByteTrack is a tracking method based on the tracking-by-detection paradigm. The detection boxes between frames are associated by similarity to obtain the tracking trajectory. The core of ByteTrack is BYTE, which can apply any detection algorithm of its own and input its own detection results into the tracker. When Yolov8 is used in combination with ByteTrack, Yolov8 can quickly and accurately detect the target in the video; at the same time, ByteTrack uses the information provided by Yolov8 to accurately track the target, which helps to solve the target tracking problem in complex scenes.

[0004] PyTorch is mainly developed for deep neural networks. As the Python version of Torch, it is designed for GPU-accelerated deep neural networks and provides an intuitive and powerful platform for building and training neural networks. The two core features of PyTorch are efficient GPU-accelerated tensor operations and automatic differentiation mechanisms, which greatly improve the efficiency and convenience of model development. PyTorch's computational graph is dynamic, allowing users to build and modify neural networks and modify computational graphs in real time.

[0005] YOLOv8 is implemented based on PyTorch. YOLOv8 is built in the PyTorch framework and inherits these advantages of PyTorch, making it a popular choice in the current field of object detection. At the same time, PyTorch's standardized documentation, easy to use, efficient, fast and convenient use, also make the development process smoother.

[0006] Today's campus car navigation systems are increasingly dependent on mobile applications, and cannot provide users with accurate location information and route guidance based on user behavior in a timely manner. In addition, the navigation system requires location permissions, which infringes on user privacy to a certain extent. In addition, for a large amount of surveillance video data, the school needs corresponding monitoring personnel to handle it and storage solutions to save it. It faces problems such as high equipment maintenance costs and insufficient monitoring personnel, and cannot quickly find illegal vehicles and improve the level of campus intelligence. Summary of the invention

[0007] Purpose of the invention: In order to solve the problems of needing positioning authority and facing some costs and insufficient monitoring personnel, the present invention provides an off-campus vehicle intelligent safety guidance method and system based on target detection technology. By collecting and analyzing user behavior data, the path planning algorithm is further optimized to improve navigation efficiency, while also improving campus management and facility layout.

[0008] After testing, the system no longer needs additional user location permissions, and the system greatly reduces manpower costs and other costs. The system can achieve partial intelligent monitoring, greatly reducing the management difficulty of campus managers and effectively improving the safety and management efficiency of campus traffic.

[0009] The present invention includes a user end and an administrator end, and the technical solution adopted to solve the technical problem includes the following steps:

[0010] (1) First, the entry and exit data filled in by off-campus vehicle users in the WeChat applet is stored, and the location information of each camera on campus is stored;

[0011] (2) Identify road condition information through a camera in the campus monitoring system, upload real-time video data to the system, use the trained model to recognize the data in real time and obtain data such as the license plate and speed of the vehicle in the video, and store and save these data in a one-to-one correspondence with the data filled in by the user;

[0012] (3) When a vehicle is identified by a camera, the location of the vehicle user’s WeChat Mini Program page is updated to the location of the camera where the vehicle is currently located. If the vehicle deviates from the original route, the system will update the user’s driving route in real time through the path algorithm;

[0013] In step (1), off-campus vehicle users enter the WeChat applet by scanning the QR code. The location information of cameras installed in various locations on campus is collected by the campus administrator and stored in a confidential manner to prevent malicious damage. The location of each camera is stored in longitude and latitude, and the weight corresponding to each camera is assigned by distance.

[0014] In step (2), the training process of generating a good model includes the following steps:

[0015] (1a) Collect images of common vehicles on campus as detection images;

[0016] (1b) Mark the collected vehicle detection images accordingly, output them as .txt files, and convert the file formats;

[0017] (1c) Divide the converted files into training set and validation set; install the YOLOv8 model and set the YOLOv8 model parameters;

[0018] (1d) Train the YOLOv8 model according to the converted training set, and output the trained YOLOv8 model;

[0019] (1e) Test the trained YOLOv8 model based on the validation set and output the accuracy N. When the accuracy meets the standard, output the model;

[0020] In step (1b), the data is labeled and output as a .txt file. In each .txt file, each line represents a separate object in the image and contains five normalized values, as follows:

[0021] (1) Object category index: an integer indicating the category to which the object belongs;

[0022] (2) The ratio of the center point of the bounding box to the image width: a floating point number ranging from 0 to 1;

[0023] (3) The ratio of the center point of the bounding box to the image height: a floating point number ranging from 0 to 1;

[0024] (4) The ratio of the bounding box width to the image width: a floating point number ranging from 0 to 1;

[0025] (5) The ratio of the bounding box height to the image height: a floating point number ranging from 0 to 1;

[0026] In the step (1c), in the generated YOLOv8 model, the Yolov8 algorithm uses a multi-part loss function to optimize positioning and classification performance.

[0027] The total loss function is shown in formula (1):

[0028] L=λ coord ·L loc +L conf +λ class ·L class (1)

[0029] λ coord and λ class is the weight coefficient used to balance the importance of different loss terms; L loc is the positioning loss; L conf is the confidence loss; L class is the classification loss.

[0030] The localization loss is shown in formula (2). The localization loss measures the gap between the predicted bounding box and the true bounding box.

[0031]

[0032] is an indicator variable, which is 1 when the jth bounding box of the i-th grid cell is responsible for predicting the target, otherwise it is 0; x, y are the coordinates of the center point of the bounding box; w, h are the width and height of the bounding box.

[0033] The confidence loss is shown in formula (3). The confidence loss measures the gap between the predicted confidence score and the actual value.

[0034]

[0035] C is the confidence score. noobj is the weight coefficient of confidence loss when there is no object; is an indicator variable, which is 1 when the j-th bounding box of the j-th grid cell is not responsible for predicting the target, otherwise it is 0;

[0036] The classification loss is shown in formula (4). The classification loss measures the gap between the predicted image category probability and the actual image category label.

[0037]

[0038] p(c) is the predicted class probability; is the true category label. is an indicator variable that is 1 if the i-th grid cell contains an object and 0 otherwise.

[0039] In the step (1e), the trained model is a Yolov8 model, and the accuracy rate N is the ratio of the number of samples correctly verified by the trained YOLOv8 model to the total number of samples in the verification set. The accuracy rate N is compared with the compliance rate N0, and the training compliance result of the trained YOLOv8 model is judged based on the comparison result.

[0040] When N ≥ N0, it is determined that the YOLOv8 model training has met the standard, and the trained YOLOv8 model is output and saved;

[0041] When N < N0, it is determined that the training of the YOLOv8 model is not up to standard, and the model training steps in step (2) are repeated until the model training is up to standard. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is the user function flow chart of the system of the present invention;

[0043] Figure 2 It is the administrator function flow chart of the present invention;

[0044] Figure 3 It is the system data flow chart of the present invention;

[0045] Figure 4 The flow chart of the path planning algorithm of the present invention;

[0046] Figure 5 The front-end page of the WeChat mini-program for the user access registration information;

[0047] Figure 6 The front-end page that automatically generates the starting point of the route for the user;

[0048] Figure 7 The front-end page that automatically generates the end point of the route for the user;

[0049] Figure 8 The front-end page for the administrator to control a single camera;

[0050] Fig. 9 It is the function module diagram of the system of the present invention: DETAILED DESCRIPTION OF THE INVENTION

[0051] The present invention will be further described in detail below with reference to the drawings and embodiments.

[0052] As Figure 1 shown is the process required for the user login front-end of the off-campus vehicle intelligent safety guidance system. For the user side, this system uses a WeChat mini-program. The input and output of the user in the WeChat mini-program collect the basic information of the user. The user generates the required route by filling in relevant information, and the route will be updated in real time through the path algorithm of this system.

[0053] As Figure 2 shown is the process required for the administrator login front-end of the off-campus vehicle intelligent safety guidance system. For the administrator side, this system uses a back-end web page. The administrator can achieve real-time monitoring of off-campus vehicles on campus roads by controlling the cameras on campus.

[0054] As Figure 3 shown is the data flow chart of the off-campus vehicle intelligent safety guidance system. The system includes two types, users and administrators. The specific process of the data flow between them is as follows:

[0055] 1. The user registers their personal information for entry and exit, uploads it to the database for storage, and completes the route guidance function through the generated route.

[0056] 2. The administrator can manage user information, view user personal information, and know whether the user is a speeding violator.

[0057] As Figure 4 shown in the flowchart of the path planning algorithm for the guidance of the off-campus vehicle intelligent safety guidance system, the specific process is as follows:

[0058] 1. Initialize by creating two sets, the CLOSED set, initially empty, for storing visited nodes, and the OPEN set, and add the starting node S to it.

[0059] 2. Enter a loop. First, check if the OPEN set is empty. If it is empty, it means the target node cannot be found and the algorithm fails. Select the node best with the smallest f value from the OPEN set. The f value is an evaluation function, usually composed of the actual cost g to reach this node and the heuristic estimated cost h, as shown in formula (5).

[0060] f = g + h (5)

[0061] 3. Put the best node into the CLOSED set, indicating that this node has been visited. Check if the best node is the target node. If it is, search backward for the previous nodes to generate the optimal path and the algorithm succeeds. If the best node is not the target node, obtain all its subsequent nodes subs. For each subsequent node sub, calculate the path length G(sub) from the starting node S through best to sub, and the calculation formula is G(sub) = g(best) + c(best,sub), where c(best,sub) is the cost of the edge from best to sub.

[0062] 4. Check if the sub node is in the OPEN set or the CLOSED set: If the sub node is not in the OPEN set and the CLOSED set, add it to the OPEN set and calculate its f value f(sub) = G(sub) + h(sub). If the sub node is in the OPEN set and the newly calculated G(sub) is less than the original g(sub), update the g value of the sub node to G(sub) and recalculate its f value. If the sub node is in the CLOSED set and G(sub) < g(sub), delete the sub node from the CLOSED set, re-add it to the OPEN set, and calculate its f value.

[0063] For this algorithm, by continuously expanding nodes, evaluating nodes, selecting the best node for search, and finally finding the best path from the start node to the target node. In each iteration, the next node to be expanded is selected based on the node's f value. The f value comprehensively considers the actual cost from the start node to the current node and the heuristic estimated cost to the target node. When processing repeated visits to nodes, the cost of the new path and the old path is compared to select a better path, ensuring that the found path is the best.

[0064] like Figure 5 The front-end picture shown is the user's entry and exit login information. Users log in through the WeChat applet to register entry and exit. Multiple input boxes need to be set to allow users to provide information such as name, mobile phone number and license plate number. Multiple picker boxes are set to allow users to select whether they are long-term entrants, destinations, entrances, and whether they agree to the security agreement. Regular expressions are set for mobile phone numbers and license plate numbers to determine whether the information format filled in by the user is correct. The submitted data is saved in the user table of the database by sending a POST request to submit the form information. Finally, if the submission is successful, jump to the route guidance page. If the submission fails, stay on this page.

[0065] like Figure 6 and Figure 7 The front-end image shown is the route generation. The campus map is visualized by introducing the Amap SDK in the JavaScript code of this page. The longitude and latitude coordinates of the address information in the database are read through a POST request. The address data previously filled in by the user is requested through a Request, and the route is visualized by reading the longitude and latitude of the address information from the database.

[0066] like Figure 8 The front page of the administrator's camera is shown below. The administrator can log in to the administrator's camera to detect campus vehicle video data. The administrator can choose to upload relevant vehicle videos or open the campus camera to detect campus vehicles. After uploading the video, the system will automatically detect it, and the detection results will be displayed on the right side of the video. It can visualize information such as whether the user is a long-term user and whether he is speeding.

[0067] like Fig. 9 The figure shows a module diagram of the system of the present invention. The system is applied to the entry and exit registration of users and the daily work of administrators, including the user end and the administrator end. The user end mainly includes the following four modules: entry and exit registration, route guidance, address information, and other content. The administrator end mainly includes the following five modules: user management, destination management, data visualization, camera management, and map visualization.

Claims

1. An intelligent safety guidance method and system for off-campus vehicles based on target detection technology, characterized in that: The following steps are involved: (1) First, the entry and exit data filled in by off-campus vehicle users in the WeChat applet is stored, and the location information of each camera on campus is stored; (2) Identify road condition information through a camera in the campus monitoring system, upload real-time video data to the system, use the trained model to recognize the data in real time and obtain data such as the license plate and speed of the vehicle in the video, and store and save these data in a one-to-one correspondence with the data filled in by the user; (3) When a vehicle is identified by a camera, the location of the vehicle user is updated on the WeChat applet page with the location of the camera where the vehicle is currently located. If the vehicle deviates from the original route, the system will update the user's driving route in real time through a path algorithm.

2. The off-campus vehicle intelligent safety guidance method and system according to claim 1 is characterized in that In step (1), off-campus vehicle users enter the WeChat applet by scanning the QR code. The location information of cameras installed in various locations on campus is collected by the campus administrator and stored in a confidential manner to prevent malicious damage.

3. According to the off-campus vehicle intelligent safety guidance method and system of claim 1, in the step (2), the training and generating model process is characterized in that: include: (1a) Collect images of common vehicles on campus as detection images; (1b) Mark the collected vehicle detection images accordingly, output them as .txt files, and convert the file formats; (1c) Divide the converted files into training set and validation set; install the YOLOv8 model and set the YOLOv8 model parameters; (1d) Train the YOLOv8 model according to the converted training set, and output the trained YOLOv8 model; (1e) Test the trained YOLOv8 model based on the validation set and output the accuracy N. When the accuracy meets the standard, output the model.

4. The training and generating model process according to claim 1, wherein in step (1b), the data labeling output is a .txt file. For each .txt file, each line represents a separate object in the image and contains five normalized values, as follows: (1) Object category index: an integer indicating the category to which the object belongs; (2) The ratio of the center point of the bounding box to the image width: a floating point number ranging from 0 to 1; (3) The ratio of the center point of the bounding box to the image height: a floating point number ranging from 0 to 1; (4) The ratio of the bounding box width to the image width: a floating point number ranging from 0 to 1; (5) The ratio of the bounding box height to the image height: a floating point number ranging from 0 to 1.

5. The training process according to claim 1 generates a good model, wherein in the step (1c), in the YOLOv8 model, in the Yolov8 official code, there are five target detection network models, namely Yolov8n, Yolov8s, Yolov8m, Yolov81, and Yolov8x. Among them, YOLOv8n is the fastest and smallest, so this patent invention selects Yolov8n as the model of the system for detecting vehicle speed and license plate.

6. In the process of training and generating a well-trained model according to claim 3, in step (1c), in the YOLOv8 model, the Yolov8 algorithm uses a multi-part loss function to optimize the localization and classification performance. The multi-part loss function is as follows: The total loss function is shown in Equation (1). L=λ coord ·L loc +L conf +λ class ·L class (1) where λ coord and λ class is the weight coefficient used to balance the importance of different loss terms; L loc is the positioning loss; L conf is the confidence loss; L class is the classification loss; The localization loss is shown in Equation (2). The localization loss measures the difference between the predicted bounding box and the ground truth bounding box. is an indicator variable, which is 1 when the jth bounding box of the i-th grid cell is responsible for predicting the target, otherwise it is 0; x, y are the coordinates of the center point of the bounding box; w, h are the width and height of the bounding box; The confidence loss is shown in Equation (3). The confidence loss measures the difference between the predicted confidence score and the actual value. C is the confidence score. noobj is the weight coefficient of confidence loss when there is no object; is an indicator variable, which is 1 when the jth bounding box of the i-th grid cell is not responsible for predicting the target, otherwise it is 0; The classification loss is shown in Equation (4). The classification loss measures the difference between the predicted class probability and the actual class label. p(c) is the predicted class probability; is the true category label. is an indicator variable that is 1 if the i-th grid cell contains an object and 0 otherwise.

7. In the process of training and generating a well-trained model according to claim 1, in step (1e), the well-trained model is the Yolov8 model. The so-called accuracy N is the ratio of the number of samples correctly verified by the trained YOLOv8 model to the total number of samples in the validation set. Compare the accuracy N with the passing rate N0, and judge the training passing result of the trained YOLOv8 model according to the comparison result. When N ≥ N0, it is determined that the training of the YOLOv8 model has passed, and the output of the YOLOv8 model is saved. When N < N0, it is determined that the training of the YOLOv8 model has not passed, and repeat the model training steps in step two until the model training passes.

8. According to the method and system for intelligent safety guidance of off-campus vehicles of claim 1, in said step (3), the path algorithm is characterized by: It includes: (3a) Obtain the road segment data of the campus network. (3b) For the road segment data, construct a complete directed campus road segment network model G(D, V) with the current position n as the directed node and the road segment as the directed arc; where D and V represent the directed node and the directed arc respectively. (3c) Initialize the parameters of the network model G(D, V), and based on the network model G(D, V), obtain all the trips EV; where initializing the parameters of the network model G(D, V) specifically includes: inputting the adjacent node matrix and setting the weight of the arc, and the weight of the arc is the distance divided by a specific value, and the weight of the arc between road segments can be set to zero to indicate that vehicles are prohibited from passing on this road segment. (3d) Set a heuristic function h(n) to estimate the weight from node n to the target node, and at the same time combine the actual weight g(n) from the starting point to node n to calculate the total weight f(n), as shown in Equation (5). f(n) = g(n) + h(n) (5) (3e) Calculate the theoretical shortest weight path of the EV trip.

Images