A vehicle motion information recognition method based on YOLO-V5

Abstract

The invention discloses a vehicle motion information recognition method based on YOLO-V5, which relates to the field of deep learning target detection, including: preprocessing video data to obtain a picture data set; distinguishing and marking the picture data set to obtain a picture data set containing a target frame; training a first YOLO-V5 model according to the picture data set containing the target frame; optimizing the first YOLO-V5 model by adding a coordinate relationship mapping mechanism, a coordinate information storage repository, an anti-misidentification mechanism, a missed detection error reporting mechanism, a dynamic and static identification mechanism, and a timing cleaning mechanism to obtain a second YOLO-V5 model; inputting the video data of the vehicle to be identified into the second YOLO-V5 model to obtain the vehicle motion information. The method has the characteristics of easy embedding, high robustness, low requirements on hardware identification, and wide applicability.

Description

Technical Field

[0001] The present invention relates to the field of deep learning target detection, and in particular to a vehicle motion information recognition method based on YOLO-V5. Background Art

[0002] The recognition of vehicle motion information (driving direction, dynamic and static status, vehicle position) has promoted the construction of smart engineering systems. Today, the construction of smart engineering systems has always been at the basic stage, especially the perception module of the external environment. The YOLO-V5 target detection algorithm is used to perceive and monitor the environment and vehicles and establish a virtual coordinate system, which greatly reduces the cost of system construction. At the same time, while ensuring the accuracy of recognition and monitoring, it also improves the robustness of the system.

[0003] The YOLO-V5 target detection algorithm is mainly used in the identification and positioning of target types, but it is impossible to detect the motion information of the target through the original method. At the same time, although the algorithm can detect the position of the target, the output of the position is based on the virtual coordinate system established by the algorithm, and it is impossible to provide specific coordinates in the world coordinate system. In addition, in terms of the setting of hyperparameters, different hyperparameters need to be set for different tasks and external conditions, especially the IOU threshold and confidence threshold in the detection process, in order to achieve high-stability recognition for specific situations. In the recognition process of YOLO-V5, the coordinates of target detection cannot be saved in real time and cleared regularly. In addition, its ultra-high FPS processing speed makes it difficult to save data. Finally, there is the misrecognition rate, which is also a common defect in the field of target detection. Measures must be taken to reduce the misrecognition rate of vehicle motion information and improve the final information detection accuracy.

[0004] To this end, how to provide a vehicle motion information recognition method based on YOLO-V5 that can provide real-world coordinates, specifically set the IOU threshold and confidence threshold in the hyperparameters, and can save in real time, clear regularly and reduce the misrecognition rate is a problem that technicians in this field urgently need to solve. Summary of the invention

[0005] In view of this, the present invention proposes a vehicle motion information recognition method based on YOLO-V5. By improving the YOLO-V5 algorithm, adding a coordinate relationship mapping mechanism, a coordinate information storage library, an anti-misidentification mechanism, a missed detection error reporting mechanism, a dynamic and static identification mechanism, and a timed cleaning mechanism, the present invention can not only provide real-world coordinates, specifically set the IOU threshold and confidence threshold in the hyperparameters, and be able to save in real time, clear regularly and reduce the misidentification rate, but also timely detect the dynamic and static status of the vehicle, whether it is in the quasi-stopping area, and issue a timely warning, and identify the distance between the vehicle and the obstacle, so that the vehicle can avoid obstacles during the journey. It has the characteristics of easy embedding, high robustness, low hardware identification requirements, and wide applicability.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A vehicle motion information recognition method based on YOLO-V5, comprising:

[0008] Step (1): Obtain video data and perform data preprocessing on the video data to obtain an image data set.

[0009] Step (2): Differentiate and label the image dataset to obtain an image dataset containing the target box.

[0010] Step (3): Train the first YOLO-V5 model based on the image dataset containing the target box.

[0011] Step (4): The first YOLO-V5 model is optimized by adding a coordinate relationship mapping mechanism, a coordinate information storage repository, an anti-misidentification mechanism, a missed detection error reporting mechanism, a motion and stillness identification mechanism, and a timed cleaning mechanism to obtain a second YOLO-V5 model.

[0012] Step (5): Input the vehicle video data to be identified into the second YOLO-V5 model to obtain vehicle motion information.

[0013] Optionally, in step (1), data preprocessing is to crop video data, and the number of cropped frames is set according to the vehicle movement speed to ensure the difference in vehicle position between pictures.

[0014] Optionally, in step (2), the distinguishing and marking is to distinguish and mark the front and body of the vehicle respectively, so as to obtain an image dataset containing a target frame of the front of the vehicle and a target frame of the body of the vehicle.

[0015] Optionally, the differentiation marking also includes differentiation marking for newly added identification types.

[0016] Optionally, set the number of training times according to the size of the image dataset containing the target box. At the same time, to ensure the speed and accuracy of recognition, select YOLO-V5l as the pre-training weight for training.

[0017] Optionally, step (3) further includes using a non-maximum suppression algorithm to perform performance evaluation on the first YOLO-V5 model, specifically:

[0018]

[0019] Among them, Pr(object) is whether the target vehicle actually appears in the frame, which is 1 if it appears and 0 if it does not. is the actual overlap ratio between the "predicted target box" and the "true target box", "pred" is the "predicted target box" obtained by the first YOLO-V5 model, and "truth" is the "true box" obtained by artificial distinction and labeling; Object_conf is the performance evaluation result, and the non-maximum suppression IOU threshold is set to 0.5. If Object_conf is greater than 0.5, it means that the performance evaluation of the first YOLO-V5 model is qualified, and the "predicted target box" is retained. If Object_conf is less than 0.5, it means that the performance evaluation of the first YOLO-V5 model is unqualified, and the "predicted target box" is removed.

[0020] Optionally, step (3) also includes setting the confidence threshold to 0.6.

[0021] Optionally, in step (4), the optimization process specifically includes:

[0022] The coordinate relationship mapping is specifically as follows: according to the coordinates of the upper left corner and the lower right corner of the target frame, the center coordinates of the target frame are calculated; the virtual coordinate system established by YOLO-V5 is compared with the world coordinate system, and a linear relationship mapping is performed on the center coordinates of the target frame and the world coordinates at the same position to obtain the real world coordinates corresponding to the center coordinates of the target frame; this facilitates the acquisition of the real coordinates of the specific position of each vehicle at any time in the entire detection system.

[0023] The coordinate information repository contains multiple target frame coordinate information repositories corresponding to different target frame types, and the target frame coordinate information repository stores the target frame center coordinates of the corresponding target frame types; so that subsequent mechanisms can obtain the corresponding target frame center coordinates from them at any time, which is convenient for the judgment of subsequent mechanisms.

[0024] The anti-misidentification mechanism is specifically as follows: before the target frame center coordinates are stored in the target frame coordinate information storage repository, the target frame coordinate information storage repository name is preset, and a judgment mechanism is used to determine whether the detected target frame name is consistent with the preset target frame coordinate information storage repository name. If they are consistent, the target frame center coordinates are stored in the corresponding target frame coordinate information storage repository.

[0025] The missed detection error reporting mechanism is as follows: detect the length in the target frame coordinate information storage library. If there is no coordinate value in the target frame coordinate information storage library, it proves that the target frame is not detected; judge the two target frame coordinate information storage libraries corresponding to the front target frame and the body target frame of the same vehicle. If the content length of one of the target frame coordinate information storage libraries has not changed during the processing of several frames, it is not recognized, and the name of the unrecognized target is prompted.

[0026] The specific static and dynamic identification mechanism is as follows: the center coordinates of the vehicle body target frame in the target frame coordinate information storage library corresponding to the vehicle body target frame are retrieved every 40 frames, and the straight-line distance between the coordinate positions of frames 1 and 40 is determined. If it is greater than a vehicle body distance, the vehicle is in a moving state, otherwise the vehicle is in a stationary state; in a stationary state, it is necessary to determine whether the vehicle is in a quasi-stopping or non-quasi-stopping area through the real-world coordinates. If it is in a non-quasi-stopping area, a timely warning is issued; the distance judgment method in the static and dynamic identification mechanism can also be used to determine the distance between the newly added identification type and the vehicle, and the distance parameters are transmitted to the control platform to control the vehicle's movement.

[0027] The specific timing cleaning mechanism is as follows: the coordinates in the coordinate information storage repository are cleaned every time 150 frames of image data are processed; memory is released in time to ensure the running and recognition efficiency of the model.

[0028] Optionally, step (5) further includes:

[0029] The direction vector composed of the center coordinates of the vehicle body target frame and the center coordinates of the vehicle head target frame output by the second YOLO-V5 model is output through the direction angle conversion algorithm, and the direction and steering angle of the direction vector are calculated and output at the same time.

[0030] The direction and steering angle of the vehicle's direction vector, the coordinates of the vehicle's specific position, and the vehicle's movement and stillness identified by the movement and stillness recognition mechanism are combined. Each frame of the detection result is processed at the same time, and two points are connected with arrows to render a direction visualization effect, which is displayed in the recognition result to represent the motion information of the vehicle in the current frame.

[0031] It can be known from the above technical solution that compared with the prior art, the present invention discloses a vehicle motion information recognition method based on YOLO-V5. The method is improved on the basis of the YOLO-V5 algorithm, and a coordinate relationship mapping mechanism, a coordinate information storage library, an anti-misidentification mechanism, a missed detection error reporting mechanism, a dynamic and static identification mechanism, and a timed cleaning mechanism are added, so that the present invention can not only provide real-world coordinates, specifically set the IOU threshold and confidence threshold in the hyperparameters, and can save in real time, clear regularly and reduce the misidentification rate, but also timely detect the dynamic and static state of the vehicle, whether it is in the quasi-stop area, and give timely warnings, and identify the distance between the vehicle and the obstacle, so that the vehicle can avoid obstacles during the journey, and has the characteristics of easy embedding, high robustness, low requirements for hardware identification, and wide applicability. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0033] Figure 1 It is a schematic diagram of the overall process of the present invention.

[0034] Figure 2 It is a schematic diagram of the YOLO-V5 network structure of the present invention.

[0035] Figure 3 It is a schematic diagram of the YOLO-V5 optimization process of the present invention.

[0036] Figure 4 It is a schematic diagram of the flow of the motion and stillness recognition mechanism of the present invention.

[0037] Figure 5 It is a schematic diagram of the direction angle conversion process of the present invention. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] The embodiment of the present invention discloses a vehicle motion information recognition method based on YOLO-V5, such as Figure 1 ,include:

[0040] Step (1): Use an external camera to capture video data of the vehicle in motion, and then save the video locally to obtain the video data; use Python to edit the video cropping program to crop the vehicle video data, and set the cropping frame number according to the vehicle's movement speed to ensure the difference in vehicle position between pictures, and obtain the picture data set.

[0041] Step (2): Use the Labelimg image labeling tool to distinguish and label the front and body of the vehicle, as well as newly added identification categories such as obstacles, to obtain an image dataset containing the vehicle front target frame, the vehicle body target frame, and the newly added identification category target frame.

[0042] Step (3): Train the first YOLO-V5 model based on the image dataset containing the target box. Set the number of training epochs according to the size of the training set. In this paper, the training set is 1,200 images, the number of vehicles is 5 (cars 1-5), the number of target types is 12 (the body and front of 5 cars and 2 obstacles), and the training is 150 epochs. At the same time, in order to ensure the speed and accuracy of recognition, YOLO-V5l is selected as the pre-training weight for training.

[0043] It also includes the performance evaluation of the first YOLO-V5 model using the non-maximum suppression algorithm, specifically:

[0044]

[0045] Among them, Pr(object) is whether the target vehicle actually appears in the frame, which is 1 if it appears and 0 if it does not. is the actual overlap ratio between the "predicted target box" and the "true target box", "pred" is the "predicted target box" obtained by the first YOLO-V5 model, and "truth" is the "true box" obtained by artificially distinguishing and labeling; Object_conf is the performance evaluation result, and the non-maximum suppression IOU threshold is set to 0.5. If Object_conf is greater than 0.5, it means that the performance evaluation of the first YOLO-V5 model is qualified, and the "predicted target box" is retained. If Object_conf is less than 0.5, it means that the performance evaluation of the first YOLO-V5 model is unqualified, and the "predicted target box" is removed; it also includes setting the confidence threshold to 0.6, which is used to perform performance evaluation on the first YOLO-V5 model to distinguish and punch in.

[0046] Step (4): The first YOLO-V5 model is optimized by adding a coordinate relationship mapping mechanism, a coordinate information storage repository, an anti-misidentification mechanism, a missed detection error reporting mechanism, a motion and stillness identification mechanism, and a timed cleaning mechanism to obtain a second YOLO-V5 model.

[0047] The optimization process specifically includes:

[0048] In the YOLO-V5 algorithm, the coordinate system recognized by the algorithm is set based on the upper left corner of the detected image as the origin and the output coordinates are relative values, but the coordinates in reality are not the coordinate values ​​directly detected by YOLO-V5; therefore, a coordinate relationship mapping is established, specifically: according to the upper left corner coordinates and the lower right corner coordinates of the target frame, the center coordinates of the target frame are calculated; the virtual coordinate system established by YOLO-V5 is compared with the world coordinate system, and a linear relationship mapping is made between the center coordinates of the target frame and the world coordinates at the same position to obtain the real world coordinates corresponding to the center coordinates of the target frame; this facilitates the acquisition of the real coordinates of the specific position of each vehicle at any time in the entire detection system.

[0049] When determining the real coordinates of the specific position of the vehicle, the center coordinates of the vehicle body target frame are selected.

[0050] During the YOLO-V5 detection process, due to its real-time detection mechanism, at least 20 images can be processed per second, but the coordinates outputted for each detection image are real-time, which means "output and deletion", and the accumulative storage of coordinates cannot be realized. Therefore, in order to realize the dynamic and static recognition of coordinates and the output of direction parameters, a coordinate information repository must be established, and for different targets, corresponding repositories need to be established so that subsequent mechanisms can obtain the corresponding target coordinates at any time, which is convenient for the judgment of subsequent mechanisms; therefore, a coordinate information repository is set, and a plurality of target frame coordinate information repositories corresponding to different target frame types are set in the coordinate information repository, and the target frame coordinate information repository stores the target frame center coordinates of the corresponding target frame types.

[0051] For target detection algorithms, misidentification is an ever-present shortcoming that cannot be avoided by increasing the data set or adjusting other hyperparameters (it can only be reduced); therefore, in order to ensure the accuracy of recognition, an anti-misidentification mechanism must be added, such as Figure 3 Specifically, before the target frame center coordinates are stored in the target frame coordinate information storage library, the target frame coordinate information storage library name is preset, and a judgment mechanism is used to determine whether the detected target frame name is consistent with the preset target frame coordinate information storage library name. If they are consistent, the target frame center coordinates are stored in the corresponding target frame coordinate information storage library.

[0052] In the YOLO-V5 target detection algorithm, the algorithm cannot accurately identify the target due to reasons such as lighting, vehicle position, and foreign object cover. At the same time, another situation is that the vehicle working in the scene is not in the field of view or is in a state of not starting work. The above situations require real-time detection by the algorithm so that the system can schedule work at any time. Therefore, a missed detection error reporting mechanism is set, such as Figure 3Specifically, the length of the target frame coordinate information storage library is detected. If there is no coordinate value in the target frame coordinate information storage library, it proves that the target frame is not detected; the two target frame coordinate information storage libraries corresponding to the front target frame and the body target frame of the same vehicle are judged. If the content length of one of the target frame coordinate information storage libraries does not change during the processing of several frames, it is not recognized, and the unrecognized target name is prompted.

[0053] Set up a motion recognition mechanism, such as Figure 4 Specifically, the center coordinates of the vehicle body target frame in the target frame coordinate information storage library corresponding to the vehicle body target frame are retrieved every 40 frames, and the straight-line distance between the coordinate positions of the 1st frame and the 40th frame is determined. If it is greater than a vehicle body distance, the vehicle is in a moving state, otherwise the vehicle is in a stationary state; in a stationary state, it is necessary to determine whether the vehicle is in an quasi-stopping or non-quasi-stopping area through the real-world coordinates. If it is in a non-quasi-stopping area, a timely warning is issued.

[0054] The distance judgment method in the dynamic and static recognition mechanism can also be used to judge new recognition types such as the distance between obstacles and vehicles, transmit distance parameters to the control platform, and control the vehicle to avoid obstacles during movement.

[0055] During the recognition process, due to its real-time nature, the coordinate values ​​in the coordinate information repository cannot be added indefinitely, and need to be cleaned up regularly to release memory in time to ensure the model's running and recognition efficiency. Therefore, the regular cleaning mechanism is set as follows: the coordinates in the coordinate information repository are cleaned up every 150 frames of image data are processed; the memory is released in time to ensure the model's running and recognition efficiency.

[0056] Step (5): Input the vehicle video data to be identified into the second YOLO-V5 model. Figure 5 , the direction vector composed of the center coordinates of the vehicle body target frame and the center coordinates of the vehicle head target frame output by the second YOLO-V5 model is output through the direction angle conversion algorithm, and the direction and steering angle of the direction vector are calculated and output at the same time, such as:

[0057] Vehicle 1 direction information: direction vector: [-16.0, -8.0];

[0058] The vehicle's current heading is: 27° west by south.

[0059] Finally, the direction and steering angle of the vehicle's direction vector, the coordinates of the vehicle's specific position, and the vehicle's movement and stillness identified by the movement and stillness recognition mechanism are combined. At the same time, each frame of the detection result is processed, and the two points are connected with arrows to render a direction visualization effect, which is displayed in the recognition result to represent the motion information of the vehicle in the current frame.

[0060] The present invention discloses a vehicle motion information recognition method based on YOLO-V5. The method is improved on the basis of the YOLO-V5 algorithm, and a coordinate relationship mapping mechanism, a coordinate information storage library, an anti-misidentification mechanism, a missed detection error reporting mechanism, a dynamic and static identification mechanism, and a timing cleaning mechanism are added, so that the present invention can not only provide real-world coordinates, specifically set the IOU threshold and confidence threshold in the hyperparameters, and can be saved in real time, cleared regularly and reduce the misidentification rate, but also can timely detect the dynamic and static state of the vehicle, whether it is in the quasi-stopping area, and give a timely warning, and identify the distance between the vehicle and the obstacle, so that the vehicle avoids the obstacle during the moving process, and has the characteristics of easy embedding, high robustness, low requirements for hardware identification, and wide applicability.

[0061] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0062] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A vehicle motion information recognition method based on YOLO-V5, characterized in that: include: Step (1): obtaining video data, and performing data preprocessing on the video data to obtain an image data set; Step (2): distinguish and mark the image data set to obtain an image data set containing a target frame; Step (3): training a first YOLO-V5 model according to the image dataset containing the target box; Step (4): Optimize the first YOLO-V5 model by adding a coordinate relationship mapping mechanism, a coordinate information storage repository, an anti-misidentification mechanism, a missed detection error reporting mechanism, a dynamic and static identification mechanism, and a timed cleaning mechanism to obtain a second YOLO-V5 model; Step (5): input the video data of the vehicle to be identified into the second YOLO-V5 model to obtain the vehicle motion information; In step (2), the distinguishing and marking is to distinguish and mark the front and body of the vehicle respectively, so as to obtain an image dataset containing a target frame of the front of the vehicle and a target frame of the body of the vehicle; The differentiation marking also includes differentiation marking of newly added identification categories; In step (4), the optimization process specifically includes: The coordinate relationship mapping is specifically as follows: according to the coordinates of the upper left corner and the lower right corner of the target frame, the center coordinates of the target frame are calculated; the virtual coordinate system established by YOLO-V5 is compared with the world coordinate system, and a linear relationship mapping is performed between the center coordinates of the target frame and the world coordinates at the same position to obtain the real world coordinates corresponding to the center coordinates of the target frame; The coordinate information storage library contains a plurality of target frame coordinate information storage libraries corresponding to different target frame types, and the target frame coordinate information storage library stores the target frame center coordinates of the corresponding target frame types; The anti-misidentification mechanism is specifically as follows: before the target frame center coordinates are stored in the target frame coordinate information storage library, a target frame coordinate information storage library name is preset, and a judgment mechanism is used to judge whether the detected target frame name is consistent with the preset target frame coordinate information storage library name, and if they are consistent, the target frame center coordinates are stored in the corresponding target frame coordinate information storage library; The missed detection error reporting mechanism is specifically as follows: detecting the length in the target frame coordinate information storage library, if there is no coordinate value in the target frame coordinate information storage library, it is proved that the target frame is not detected; judging the two target frame coordinate information storage libraries corresponding to the front target frame and the body target frame of the same vehicle, if the content length of one of the target frame coordinate information storage libraries does not change during the processing of several frames, it is not recognized, and prompting the name of the unrecognized target; The dynamic and static identification mechanism is specifically as follows: the center coordinates of the vehicle body target frame in the target frame coordinate information storage library corresponding to the vehicle body target frame are retrieved every 40 frames, and the straight-line distance between the coordinate positions of the 1st frame and the 40th frame is determined. If the straight-line distance is greater than a vehicle body distance, the vehicle is in a moving state, otherwise the vehicle is in a stationary state; In a stationary state, the real-world coordinates are used to determine whether the vehicle is in an accurate parking area or a non-accurate parking area. If the vehicle is in a non-accurate parking area, a warning is issued in a timely manner; The timing cleaning mechanism is specifically: cleaning the coordinates in the coordinate information storage library every time 150 frames of image data are processed.

2. A vehicle motion information recognition method based on YOLO-V5 according to claim 1, characterized in that: In step (1), the data preprocessing is to crop the video data, and the number of cropped frames is set according to the vehicle movement speed.

3. The vehicle motion information recognition method based on YOLO-V5 according to claim 1, characterized in that: In step (3), the number of training times is set according to the size of the image data set containing the target frame.

4. The vehicle motion information recognition method based on YOLO-V5 according to claim 1, characterized in that: Step (3) also includes using a non-maximum suppression algorithm to perform performance evaluation on the first YOLO-V5 model, specifically: Among them, Pr(object) is whether the target vehicle actually appears in the frame, which is 1 if it appears and 0 if it does not. is the actual overlap ratio between the "predicted target box" and the "true target box", "pred" is the "predicted target box" obtained by the first YOLO-V5 model, and "truth" is the "true box" obtained by artificial distinction and labeling; Object_conf is the performance evaluation result, and the non-maximum suppression IOU threshold is set to 0.

5. If Object_conf is greater than 0.5, it means that the performance evaluation of the first YOLO-V5 model is qualified, and the "predicted target box" is retained. If Object_conf is less than 0.5, it means that the performance evaluation of the first YOLO-V5 model is unqualified, and the "predicted target box" is removed.

5. The vehicle motion information recognition method based on YOLO-V5 according to claim 1, characterized in that: Step (5) further includes: The direction vector composed of the center coordinates of the vehicle body target frame and the center coordinates of the vehicle head target frame output by the second YOLO-V5 model is output through a direction angle conversion algorithm, and the direction and steering angle of the direction vector are calculated and output at the same time.

Images