Road area calibration method and electronic equipment
Through the object detection neural network model and matching algorithm to identify the vehicle motion trajectory in road video, the problem of vehicle-intensive area calibration in the prior art is solved and the efficiency of traffic travel is improved.
Patent Information
- Application Number
- CN202111006532.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-08-30
- Publication Date
- 2024-12-10
- Estimated Expiration
- 2041-08-30
AI Technical Summary
The lack of effective methods in the prior art to calibrate vehicle-intensive areas in urban roads, resulting in low traffic travel efficiency.
The preset object detection neural network model is used to detect vehicle targets on road videos, identify vehicles by detecting positions and motion trajectories, match predicted positions using Hungarian algorithms and Kalman filtering algorithms, and determine vehicles with non-maximum suppression and clustering algorithms. Dense areas.
Accurate calibration and intervention in densely packed areas of vehicles have been achieved, and traffic travel efficiency has been improved.
Smart Images
Figure CN113763425B_ABST
Abstract
Description
Background Art
[0002] With the rapid development of economy, there are more and more vehicles on urban roads, which leads to more and more congested urban traffic. Therefore, it is necessary to calibrate the densely populated areas on urban traffic roads so as to intervene in the densely populated areas in advance, so as to improve the travel efficiency of vehicles.
[0003] However, there is no method in the prior art for calibrating the area with dense vehicles on the road, which results in low efficiency of traffic travel. Summary of the invention
[0004] In an exemplary embodiment of the present disclosure, a road area calibration method and an electronic device are provided for calibrating a vehicle-dense area on a road, so as to intervene in the vehicle-dense area in advance and improve the efficiency of traffic travel.
[0005] A first aspect of the present disclosure provides a road area calibration method, the method comprising:
[0006] For any frame of video image within a specified time length in the road video, a preset target detection neural network model is used to perform vehicle target detection on the video image to obtain the detection position of each target vehicle;
[0007] Identify each target vehicle based on the detected position of each target vehicle to obtain an identification of each target vehicle, wherein the identification of the same target vehicle in different video images is the same;
[0008] For any target vehicle, the movement trajectory of the target vehicle on the road is determined by the identification and detection position of the target vehicle in each frame of the video image within the specified time period;
[0009] According to the movement trajectory of each target vehicle on the road, a vehicle dense area on the road is determined.
[0010] In this embodiment, by using a preset target detection neural network model to detect vehicle targets on any frame of video image within a specified time length in the road video, the detection position of each target vehicle is obtained, and then each target vehicle is identified based on the detection position of each target vehicle to obtain the identification of each target vehicle, and for any target vehicle, the movement trajectory of the target vehicle in the road is determined by the identification and detection position of the target vehicle in each frame of video image within the specified time length; finally, the vehicle-dense area on the road is determined according to the movement trajectory of each target vehicle on the road. Therefore, this embodiment can determine the vehicle-dense area on the road within the specified time length, and can timely intervene in the vehicle-dense area on the road to improve vehicle travel efficiency.
[0011] In one embodiment, the dilation rate of at least one target convolution in the backbone network of the target detection neural network model is a hole convolution with a set value, and the target convolution is a convolution of a specified size.
[0012] This embodiment replaces at least one convolution of a specified size in the backbone network of the target detection neural network model with a dilated convolution with a set dilation rate. In this way, the dilated convolution in the target detection neural network is used to expand the global field of view to obtain more abstract features and spatial information in the video image, making the vehicle target detection result more accurate.
[0013] In one embodiment, the identifying each target vehicle based on the detected position of each target vehicle to obtain the identification of each target vehicle includes:
[0014] For any target vehicle, the detected position of the target vehicle is matched with the predicted position of each target vehicle respectively by using the Hungarian algorithm to determine the predicted position corresponding to the detected position of the target vehicle; wherein the predicted position of each target vehicle is obtained by predicting the detected position of each target vehicle in the previous frame of video image by using the Kalman filter algorithm;
[0015] If it is determined that the number of predicted positions corresponding to the detected position of the target vehicle is one, determining the identifier of the target vehicle in the previous frame of video image corresponding to the predicted position as the identifier of the target vehicle;
[0016] If it is determined that there are multiple predicted positions corresponding to the detected position of the target vehicle, the predicted position with the highest priority is determined based on the cascade matching parameters corresponding to each predicted position, and the identification of the target vehicle in the previous frame of the video image corresponding to the predicted position with the highest priority is determined as the identification of the target vehicle, wherein the cascade matching parameters are used to indicate the priority of the predicted position.
[0017] In this embodiment, the detection position of the target vehicle is matched with the predicted position of each target vehicle respectively through the Hungarian algorithm to determine the predicted position corresponding to the detection position of the target vehicle, and when it is determined that there are multiple predicted positions corresponding to the detection position of the target vehicle, the predicted position with the highest priority is determined based on the cascade matching parameters corresponding to each predicted position, and the identification of the target vehicle in the previous frame of the video image corresponding to the predicted position with the highest priority is determined as the identification of the target vehicle, so that the determined identification of the target vehicle is more accurate.
[0018] In one embodiment, for any target vehicle, the detection position of the target vehicle is matched with the predicted position of each target vehicle respectively by using the Hungarian algorithm, and before determining the predicted position corresponding to the detection position of the target vehicle, the method further includes:
[0019] For each target vehicle having a plurality of detection positions, the detection positions of each target vehicle are screened based on a preset algorithm to obtain the screened detection positions of each target vehicle.
[0020] In this embodiment, the detection position of the target vehicle is matched with the predicted position of each target vehicle respectively through the Hungarian algorithm. Before determining the predicted position corresponding to the detection position of the target vehicle, it is necessary to screen the multiple detection positions of each target vehicle with multiple detection positions to ensure the accuracy of the matching result.
[0021] In one embodiment, the detection positions of each target vehicle are screened based on a preset algorithm to obtain the screened detection positions of each target vehicle, including:
[0022] After the detection position of each target vehicle is screened by using a non-maximum suppression algorithm, the detection position of each target vehicle is screened again by the confidence of the detection position of each target vehicle to obtain the screened detection position of each target vehicle; wherein the confidence of the detection position of each target vehicle is obtained based on a preset target detection neural network model.
[0023] In this embodiment, after the detection position of each target vehicle is screened by using the non-maximum suppression algorithm, the detection position of each target vehicle is screened again according to the confidence of the detection position of each target vehicle to obtain the detection position of each target vehicle after screening. Through two screenings, it is ensured that the detection position of each target vehicle after screening is more accurate.
[0024] In one embodiment, determining a vehicle dense area on the road according to the movement trajectory of each target vehicle on the road includes:
[0025] Divide the motion trajectory of each target vehicle into a specified number of sub-motion trajectories, wherein the lengths of the sub-motion trajectories in the same motion trajectory are equal;
[0026] Clustering each sub-motion trajectory using a preset clustering algorithm to obtain multiple motion trajectory clustering areas;
[0027] For any motion trajectory clustering area, if it is determined that the number of each sub-motion trajectory in the motion trajectory clustering area is greater than a specified threshold, the motion trajectory clustering area is determined to be a vehicle dense area on the road.
[0028] In this embodiment, the motion trajectory of each target vehicle is divided into a specified number of sub-motion trajectories of equal length, and each sub-motion trajectory is clustered using a preset clustering algorithm to obtain multiple motion trajectory clustering areas. For any motion trajectory clustering area, if it is determined that the number of each sub-motion trajectory in the motion trajectory clustering area is greater than a specified threshold, the motion trajectory clustering area is determined to be a vehicle-dense area on the road. Therefore, by dividing the motion trajectory into multiple sub-motion trajectories, the clustering process is ensured to be more fine-grained, so that the obtained motion trajectory clustering area is more accurate.
[0029] A second aspect of the present disclosure provides an electronic device, including a storage unit and a processor, wherein:
[0030] The storage unit is configured to store the road video;
[0031] The processor is configured to:
[0032] For any frame of video image within a specified time length in the road video, a preset target detection neural network model is used to perform vehicle target detection on the video image to obtain the detection position of each target vehicle;
[0033] Identify each target vehicle based on the detected position of each target vehicle to obtain an identification of each target vehicle, wherein the identification of the same target vehicle in different video images is the same;
[0034] For any target vehicle, the movement trajectory of the target vehicle on the road is determined by the identification and detection position of the target vehicle in each frame of the video image within the specified time period;
[0035] According to the movement trajectory of each target vehicle on the road, a vehicle dense area on the road is determined.
[0036] In one embodiment, the dilation rate of at least one target convolution in the backbone network of the target detection neural network model is a hole convolution with a set value, and the target convolution is a convolution of a specified size.
[0037] In one embodiment, the processor performs the identification of each target vehicle based on the detected position of each target vehicle to obtain the identification of each target vehicle, which is specifically configured as follows:
[0038] For any target vehicle, the detected position of the target vehicle is matched with the predicted position of each target vehicle respectively by using the Hungarian algorithm to determine the predicted position corresponding to the detected position of the target vehicle; wherein the predicted position of each target vehicle is obtained by predicting the detected position of each target vehicle in the previous frame of video image by using the Kalman filter algorithm;
[0039] If it is determined that the number of predicted positions corresponding to the detected position of the target vehicle is one, determining the identifier of the target vehicle in the previous frame of video image corresponding to the predicted position as the identifier of the target vehicle;
[0040] If it is determined that there are multiple predicted positions corresponding to the detected position of the target vehicle, the predicted position with the highest priority is determined based on the cascade matching parameters corresponding to each predicted position, and the identification of the target vehicle in the previous frame of the video image corresponding to the predicted position with the highest priority is determined as the identification of the target vehicle, wherein the cascade matching parameters are used to indicate the priority of the predicted position.
[0041] In one embodiment, the processor is further configured to:
[0042] For any target vehicle, the detection position of the target vehicle is matched with the predicted positions of each target vehicle respectively by using the Hungarian algorithm. Before determining the predicted position corresponding to the detection position of the target vehicle, for each target vehicle having a plurality of detection positions, the detection position of each target vehicle is screened based on a preset algorithm to obtain the screened detection positions of each target vehicle.
[0043] In one embodiment, the processor performs the screening of the detection positions of each target vehicle based on a preset algorithm to obtain the screened detection positions of each target vehicle, which is specifically configured as follows:
[0044] After the detection position of each target vehicle is screened by using a non-maximum suppression algorithm, the detection position of each target vehicle is screened again by the confidence of the detection position of each target vehicle to obtain the screened detection position of each target vehicle; wherein the confidence of the detection position of each target vehicle is obtained based on a preset target detection neural network model.
[0045] In one embodiment, the processor performs the step of determining the vehicle-dense area on the road according to the motion trajectory of each target vehicle on the road, and is specifically configured as follows:
[0046] Divide the motion trajectory of each target vehicle into a specified number of sub-motion trajectories, wherein the lengths of the sub-motion trajectories in the same motion trajectory are equal;
[0047] Clustering each sub-motion trajectory using a preset clustering algorithm to obtain multiple motion trajectory clustering areas;
[0048] For any motion trajectory clustering area, if it is determined that the number of each sub-motion trajectory in the motion trajectory clustering area is greater than a specified threshold, the motion trajectory clustering area is determined to be a vehicle dense area on the road. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0050] Figure 1 A schematic diagram of an applicable scenario in an embodiment of the present disclosure;
[0051] Figure 2 This is one of the flowcharts of a road area calibration method according to an embodiment of the present disclosure;
[0052] Figure 3 The second schematic diagram of the dilated convolution of the road area calibration method according to an embodiment of the present disclosure;
[0053] Figure 4 It is a structural schematic diagram of a target detection neural network model refinedet in a road area calibration method according to an embodiment of the present disclosure;
[0054] Figure 5 A schematic diagram of vehicle target detection in a road area calibration method according to an embodiment of the present disclosure;
[0055] Figure 6 A schematic diagram of target vehicle position screening in a road area calibration method according to an embodiment of the present disclosure;
[0056] Figure 7 A schematic diagram of a target vehicle identification in a road area calibration method according to an embodiment of the present disclosure;
[0057] Figure 8 A schematic diagram of a target vehicle trajectory in a road area calibration method according to an embodiment of the present disclosure;
[0058] Fig. 9 A schematic diagram of a process for determining a vehicle-dense area in a road area calibration method according to an embodiment of the present disclosure;
[0059] Fig.10 A schematic diagram of a process for determining a vehicle-dense area according to an embodiment of the present disclosure;
[0060] Fig.11The second flowchart of the road area calibration method according to an embodiment of the present disclosure is as follows;
[0061] Fig.12 A road area demarcation device according to an embodiment of the present disclosure;
[0062] Fig.13 The figure is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0063] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.
[0064] In the embodiments of the present disclosure, the term "and / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent three situations: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.
[0065] The application scenarios described in the embodiments of the present disclosure are intended to more clearly illustrate the technical solutions of the embodiments of the present disclosure, and do not constitute a limitation on the technical solutions provided by the embodiments of the present disclosure. It is known to those skilled in the art that with the emergence of new application scenarios, the technical solutions provided by the embodiments of the present disclosure are also applicable to similar technical problems. In the description of the present disclosure, unless otherwise specified, the meaning of "multiple" is two or more.
[0066] There is no method in the prior art for calibrating the area with dense vehicles on the road, which results in low efficiency of traffic travel.
[0067] Therefore, the present disclosure provides a road area calibration method, which performs vehicle target detection on any frame of video image within a specified time length in a road video using a preset target detection neural network model to obtain the detection position of each target vehicle, and then identifies each target vehicle based on the detection position of each target vehicle to obtain the identification of each target vehicle, and for any target vehicle, determines the movement trajectory of the target vehicle in the road through the identification and detection position of the target vehicle in each frame of video image within the specified time length; finally, determines the vehicle-dense area in the road according to the movement trajectory of each target vehicle in the road. Therefore, this embodiment can determine the vehicle-dense area in the road within the specified time length, and can timely intervene in the vehicle-dense area in the road to improve vehicle travel efficiency. Below, the scheme of the present disclosure is described in detail in conjunction with the accompanying drawings.
[0068] like Figure 1 As shown, an application scenario of a road area calibration method includes a camera 110, a server 120 and a terminal device 130. Figure 1 In the figure, a camera 110 and a terminal device 130 are taken as an example. In fact, the number of cameras 110 and terminal devices 130 is not limited. The terminal device 130 can be a mobile phone, a tablet computer, a personal computer, etc. The server 120 can be implemented by a single server or multiple servers. The server 120 can be implemented by a physical server or a virtual server.
[0069] In a possible application scenario, the camera 110 sends the acquired road video to the server 120. The server 120 performs vehicle target detection on any frame of video image within a specified time length in the received road video using a preset target detection neural network model to obtain the detection position of each target vehicle; and identifies each target vehicle based on the detection position of each target vehicle to obtain the identification of each target vehicle, wherein the identification of the same target vehicle in different video images is the same; then the server 120 determines the movement trajectory of any target vehicle on the road through the identification and detection position of the target vehicle in each frame of video image within the specified time length; finally, the server 120 determines the vehicle-dense area on the road according to the movement trajectory of each target vehicle on the road, and sends the vehicle-dense area on the road to the terminal device 130 for display.
[0070] Figure 2 The flowchart of the road area calibration method disclosed in the present invention may include the following steps:
[0071] Step 201: for any frame of video image within a specified time length in a road video, a preset target detection neural network model is used to perform vehicle target detection on the video image to obtain the detection position of each target vehicle;
[0072] In order to make the vehicle target detection result more accurate, in one embodiment, the expansion rate of at least one target convolution in the backbone network of the target detection neural network model is a hole convolution with a set value, and the target convolution is a convolution of a specified size.
[0073] It should be noted that the expansion rate in this embodiment is greater than 1.
[0074] Take the convolution of the specified size as 3*3 as an example. Figure 3 As shown, Figure 3 Image 3a in FIG. 1 is a schematic diagram of a 3*3 convolution operation in a target detection neural network model in the prior art. As can be seen from Image 3a, the receptive field of the 3*3 convolution operation in the prior art is 3*3. Figure 3 As shown in the image 3b in FIG, the receptive field of the convolution operation using the same size of 3*3 and the dilation rate of 2 is 5*5. Therefore, the improved target detection neural network in this embodiment expands the global field of view to obtain more abstract features and spatial information in the video image, further improving the accuracy of the vehicle target detection result.
[0075] Next, taking the target detection neural network model as refinedet as an example, the vehicle target detection process in the present disclosure is described in detail. Figure 4 As shown in FIG. 1 , it is a schematic diagram of the structure of the improved refinedet, which includes ARM (Anchor Refinement Module), TCB (Transfer Connection Block) and ODM (Object Detection Module). Among them, the backbone network of refinedet is ARM. This embodiment takes the replacement of the second convolution in ARM with a dilation rate of 2 as an example for explanation:
[0076] First, the video image is first convolved through multiple layers in the ARM to obtain feature maps of different sizes. In the ARM, the position and score of the vehicle can be roughly estimated, and the invalid positions of some vehicles can be filtered out to reduce the search space of the classifier and roughly adjust the positions of the remaining vehicles. At the same time, the feature map obtained by the ARM is input into the ODM through the TCB, and the high-level feature map in each feature map obtained by the ARM is fused with the underlying feature map to enhance the semantic information of the underlying feature map in order to detect smaller vehicle targets. In this way, the position of each target vehicle is obtained. Figure 5 As shown, image 5a is the video image to be detected, and image 5b is the video image after vehicle target detection, and the video image includes the position of each target vehicle.
[0077] like Figure 5 As shown, since there may be multiple positions of each target vehicle, in order to ensure the accuracy of the recognition result, in one embodiment, for each target vehicle with multiple detection positions, the detection position of each target vehicle is screened based on a preset algorithm to obtain the screened detection position of each target vehicle.
[0078] in Figure 5 Image 5b in the figure is an image after vehicle target detection. As shown in image 5b, there are multiple bounding boxes of two vehicles, that is, the number of locations where two vehicles exist is multiple. Therefore, the bounding boxes of the two vehicles need to be screened, that is, after screening, Figure 6 shown.
[0079] In one embodiment, the detection position of each target vehicle is screened in the following manner:
[0080] After the detection position of each target vehicle is screened by using a non-maximum suppression algorithm, the detection position of each target vehicle is screened again by the confidence of the detection position of each target vehicle to obtain the screened detection position of each target vehicle; wherein the confidence of the detection position of each target vehicle is obtained based on a preset target detection neural network model.
[0081] Among them, the overall process of using the non-maximum suppression algorithm to screen the detection positions of each target vehicle is: for any target vehicle, sort the detection positions according to the confidence of each detection position of the target vehicle, determine the detection position with the highest confidence among the detection positions of the target vehicle, and perform IOU (Intersection-over-Union) calculation on the detection positions of the target vehicle except the detection position with the highest confidence, delete the detection positions with IOU values greater than the specified threshold, and then determine whether the number of remaining detection positions is within the specified range. If not, return to execute the step of sorting the detection positions according to the confidence of each detection position of the target vehicle and determining the detection position with the highest confidence among the detection positions of the target vehicle, until the number of remaining detection positions is within the specified range, then end.
[0082] Since the target in this embodiment is a vehicle, some of them may be similar in shape to vehicles, and the target detection neural network model may also mistakenly identify them as vehicles. Therefore, it is necessary to delete the detection positions of each target vehicle whose confidence level is less than a specified threshold. Since each target vehicle has only one corresponding position after screening using the non-maximum suppression algorithm, after deleting the position, the object at that position is no longer the target vehicle in this embodiment, thereby further improving the accuracy of vehicle recognition.
[0083] Step 202: identifying each target vehicle based on the detected position of each target vehicle to obtain an identification of each target vehicle, wherein the identification of the same target vehicle in different video images is the same;
[0084] In one implementation, the identity of each target vehicle may be determined by:
[0085] For any target vehicle, the detection position of the target vehicle is matched with the predicted positions of each target vehicle respectively by the Hungarian algorithm to determine the predicted position corresponding to the detection position of the target vehicle; wherein, the predicted position of each target vehicle is obtained by predicting the detection position of each target vehicle in the previous frame of video image by using the Kalman filter algorithm; if it is determined that the number of predicted positions corresponding to the detection position of the target vehicle is one, then the identification of the target vehicle in the previous frame of video image corresponding to the predicted position is determined as the identification of the target vehicle; if it is determined that the number of predicted positions corresponding to the detection position of the target vehicle is multiple, then the highest priority prediction position is determined based on the cascade matching parameters corresponding to each prediction position, and the identification of the target vehicle in the previous frame of video image corresponding to the highest priority prediction position is determined as the identification of the target vehicle, wherein the cascade matching parameters are used to indicate the priority of the prediction position.
[0086] For example, if there are three predicted positions (bounding boxes) corresponding to a certain vehicle, namely bounding box 1, bounding box 2 and bounding box 3, the initial priority of the bounding boxes is: bounding box 1> bounding box 2> bounding box 3., and the cascade matching parameters corresponding to bounding box 1, bounding box 2 and bounding box 3 are: 1, 0 and 2 respectively. The larger the cascade matching parameter, the lower the priority of the corresponding bounding box. It can be seen that the cascade priority of the bounding box is less than the cascade priority of the bounding box, so it is necessary to adjust the initial priority of each bounding box. The adjusted priorities of each bounding box are: bounding box 2> bounding box 1> bounding box 3. Therefore, it is determined that the priority of bounding box 2 is the highest, so the identifier of the target vehicle in the previous frame of the video image corresponding to the bounding box is determined as the identifier of the target vehicle. If the identifier of the target vehicle in the previous frame of the video image corresponding to bounding box 2 is vehicle 2, the identifier of the vehicle is set to vehicle 2.
[0087] like Figure 7 As shown, the current frame video image is the nth frame video image, and the nth frame video image is a video image obtained after identifying each target vehicle based on the detection position of each target vehicle. Figure 7 It can be seen that the nth frame of video image includes the identification of each target vehicle. The identification of each target vehicle in the nth frame of video image corresponds to the identification of each target vehicle in the 1st frame and the n-1th frame.
[0088] Step 203: for any target vehicle, determine the movement trajectory of the target vehicle on the road through the identification and detection position of the target vehicle in each frame of video image within the specified time period;
[0089] Since each frame of video image within the specified time includes the detection position and identification of each target vehicle, the motion trajectory of each target vehicle can be obtained, such as Figure 8 As shown, the motion trajectories of vehicle 1, vehicle 2, vehicle 3 and vehicle n are obtained.
[0090] Step 204: Determine a vehicle-dense area on the road according to the movement trajectory of each target vehicle on the road.
[0091] In one embodiment, Fig. 9 As shown, a schematic diagram of a process for determining a densely populated vehicle area may include the following steps:
[0092] Step 901: Divide the motion trajectory of each target vehicle into a specified number of sub-motion trajectories, wherein the lengths of the sub-motion trajectories in the same motion trajectory are equal;
[0093] Step 902: clustering each sub-motion trajectory using a preset clustering algorithm to obtain a plurality of motion trajectory clustering regions;
[0094] Step 903: for any motion trajectory clustering area, if it is determined that the number of each sub-motion trajectory in the motion trajectory clustering area is greater than a specified threshold, then the motion trajectory clustering area is determined to be a vehicle dense area on the road.
[0095] For example, Fig.10 The image 10a in FIG. Figure 8 The sub-motion trajectories are obtained by dividing the motions in the image, and then each sub-motion trajectory is represented by the center point of the sub-motion trajectory, that is, image 10b. Then, the points corresponding to each sub-motion trajectory are clustered using a preset clustering algorithm to obtain multiple motion trajectory clustering areas, as shown in image 10c, namely area 1, area 2 and area 3. If the number of each sub-motion trajectory in area 2 is greater than the specified threshold, it can be determined that area 2 is a vehicle-dense area. Image 10c is the final displayed image.
[0096] In order to further understand the technical solution of the present disclosure, Fig.11 A detailed description may include the following steps:
[0097] Step 1101: for any frame of video image within a specified time length in a road video, a preset target detection neural network model is used to perform vehicle target detection on the video image to obtain the detection position of each target vehicle;
[0098] Step 1102: After the detection positions of each target vehicle are screened using the non-maximum suppression algorithm, the detection positions of each target vehicle are screened again using the confidence of the detection positions of each target vehicle to obtain the screened detection positions of each target vehicle;
[0099] Step 1103: for any target vehicle, the detected position of the target vehicle is matched with the predicted positions of each target vehicle respectively by using the Hungarian algorithm to determine the predicted position corresponding to the detected position of the target vehicle; wherein the predicted position of each target vehicle is obtained by predicting the detected position of each target vehicle in the previous frame of video image by using the Kalman filter algorithm;
[0100] Step 1104: determine whether the number of predicted positions corresponding to the detected position of the target vehicle is multiple, if yes, execute step 1105, if no, execute step 1106;
[0101] Step 1105: determining the prediction position with the highest priority based on the cascade matching parameters corresponding to each prediction position, and determining the identifier of the target vehicle in the previous frame of the video image corresponding to the prediction position with the highest priority as the identifier of the target vehicle, wherein the cascade matching parameters are used to indicate the priority of the prediction position;
[0102] Step 1106: determining the identifier of the target vehicle in the previous frame of video image corresponding to the predicted position as the identifier of the target vehicle;
[0103] Step 1107: for any target vehicle, determine the movement trajectory of the target vehicle on the road through the identification and detection position of the target vehicle in each frame of video image within the specified time period;
[0104] Step 1108: Divide the motion trajectory of each target vehicle into a specified number of sub-motion trajectories, wherein the lengths of the sub-motion trajectories in the same motion trajectory are equal;
[0105] Step 1109: clustering each sub-motion trajectory using a preset clustering algorithm to obtain a plurality of motion trajectory clustering regions;
[0106] Step 1110: clustering each sub-motion trajectory using a preset clustering algorithm to obtain a plurality of motion trajectory clustering regions.
[0107] Based on the same disclosed concept, the road area calibration method described above in the present disclosure can also be implemented by a road area calibration device. The effect of the road area calibration device is similar to that of the aforementioned method, and will not be described in detail here.
[0108] Fig.12 The figure is a schematic diagram of the structure of a road area calibration device according to an embodiment of the present disclosure.
[0109] like Fig.12 As shown, the road area calibration device 1200 of the present disclosure may include a vehicle target detection module 1210 , a target vehicle recognition module 1220 , a motion trajectory determination module 1230 and a vehicle dense area determination module 1240 .
[0110] The vehicle target detection module 1210 is used to perform vehicle target detection on any frame of video image within a specified time length in the road video using a preset target detection neural network model to obtain the detection position of each target vehicle;
[0111] The target vehicle identification module 1220 is used to identify each target vehicle based on the detection position of each target vehicle to obtain the identification of each target vehicle, wherein the identification of the same target vehicle in different video images is the same;
[0112] The motion trajectory determination module 1230 is used to determine the motion trajectory of any target vehicle on the road through the identification and detection position of the target vehicle in each frame of the video image within the specified time period;
[0113] The vehicle-dense area determination module 1240 is used to determine the vehicle-dense area on the road according to the movement trajectory of each target vehicle on the road.
[0114] In one embodiment, the dilation rate of at least one target convolution in the backbone network of the target detection neural network model is a hole convolution with a set value, and the target convolution is a convolution of a specified size.
[0115] In one embodiment, the target vehicle identification module 1220 is specifically used to:
[0116] For any target vehicle, the detected position of the target vehicle is matched with the predicted position of each target vehicle respectively by using the Hungarian algorithm to determine the predicted position corresponding to the detected position of the target vehicle; wherein the predicted position of each target vehicle is obtained by predicting the detected position of each target vehicle in the previous frame of video image by using the Kalman filter algorithm;
[0117] If it is determined that the number of predicted positions corresponding to the detected position of the target vehicle is one, determining the identifier of the target vehicle in the previous frame of video image corresponding to the predicted position as the identifier of the target vehicle;
[0118] If it is determined that there are multiple predicted positions corresponding to the detected position of the target vehicle, the predicted position with the highest priority is determined based on the cascade matching parameters corresponding to each predicted position, and the identification of the target vehicle in the previous frame of the video image corresponding to the predicted position with the highest priority is determined as the identification of the target vehicle, wherein the cascade matching parameters are used to indicate the priority of the predicted position.
[0119] In one embodiment, the apparatus further comprises:
[0120] The screening module 1250 is used for matching the detection position of any target vehicle with the predicted positions of each target vehicle respectively by using the Hungarian algorithm, and before determining the predicted position corresponding to the detection position of the target vehicle, for each target vehicle having a plurality of detection positions, screening the detection positions of each target vehicle based on a preset algorithm to obtain the screened detection positions of each target vehicle.
[0121] In one embodiment, the screening module 1250 is specifically used to:
[0122] After the detection position of each target vehicle is screened by using a non-maximum suppression algorithm, the detection position of each target vehicle is screened again by the confidence of the detection position of each target vehicle to obtain the screened detection position of each target vehicle; wherein the confidence of the detection position of each target vehicle is obtained based on a preset target detection neural network model.
[0123] In one embodiment, the vehicle dense area determination module 1240 is specifically used to:
[0124] Divide the motion trajectory of each target vehicle into a specified number of sub-motion trajectories, wherein the lengths of the sub-motion trajectories in the same motion trajectory are equal;
[0125] Clustering each sub-motion trajectory using a preset clustering algorithm to obtain multiple motion trajectory clustering areas;
[0126] For any motion trajectory clustering area, if it is determined that the number of each sub-motion trajectory in the motion trajectory clustering area is greater than a specified threshold, the motion trajectory clustering area is determined to be a vehicle dense area on the road.
[0127] After introducing a road area demarcation method and device according to an exemplary embodiment of the present disclosure, next, an electronic device according to another exemplary embodiment of the present disclosure is introduced.
[0128] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".
[0129] In some possible implementations, the electronic device according to the present disclosure may include at least one processor and at least one computer storage medium. The computer storage medium stores program code, and when the program code is executed by the processor, the processor executes the steps of the road area calibration method according to various exemplary embodiments of the present disclosure described above in this specification. For example, the processor may execute the following steps: Figure 2 Steps 201-204 shown in .
[0130] Refer to the following Fig.13 1300 according to this embodiment of the present disclosure is described. Fig.13 The electronic device 1300 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0131] like Fig.13 As shown, the electronic device 1300 is in the form of a general electronic device. The components of the electronic device 1300 may include but are not limited to: the at least one processor 1301, the at least one computer storage medium 1302, and a bus 1303 connecting different system components (including the computer storage medium 1302 and the processor 1301).
[0132] Bus 1303 represents one or more of several types of bus structures, including a computer storage media bus or computer storage media controller, a peripheral bus, a processor, or a local bus using any of a variety of bus architectures.
[0133] Computer storage media 1302 may include readable media in the form of volatile computer storage media, such as random access computer storage media (RAM) 1321 and / or cache storage media 1322 , and may further include read-only computer storage media (ROM) 1323 .
[0134] The computer storage medium 1302 may also include a program / utility 1325 having a set (at least one) of program modules 1324, such program modules 1324 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.
[0135] The electronic device 1300 may also communicate with one or more external devices 1304 (e.g., keyboards, pointing devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 1300, and / or communicate with any device that enables the electronic device 1300 to communicate with one or more other electronic devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 1305. Furthermore, the electronic device 1300 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 1306. As shown, the network adapter 1306 communicates with other modules for the electronic device 1300 via a bus 1303. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 1300, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0136] In some possible implementations, various aspects of a road area calibration method provided by the present disclosure may also be implemented in the form of a program product, which includes program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps of the road area calibration method according to various exemplary embodiments of the present disclosure described above in this specification.
[0137] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access computer storage medium (RAM), a read-only computer storage medium (ROM), an erasable programmable read-only computer storage medium (EPROM or flash memory), an optical fiber, a portable compact disk read-only computer storage medium (CD-ROM), an optical computer storage medium, a magnetic computer storage medium, or any suitable combination of the above.
[0138] The program product of the road area demarcation of the embodiment of the present disclosure may adopt a portable compact disk read-only computer storage medium (CD-ROM) and include program code, and can be run on an electronic device. However, the program product of the present disclosure is not limited thereto, and in this document, a readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device.
[0139] The readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, wherein the readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.
[0140] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.
[0141] Program code for performing the disclosed operations may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user electronic device, partially on the user device, as a separate software package, partially on the user electronic device and partially on a remote electronic device, or entirely on a remote electronic device or server. In the case of a remote electronic device, the remote electronic device may be connected to the user electronic device via any type of network including a local area network (LAN) or a wide area network (WAN), or may be connected to an external electronic device (e.g., via the Internet using an Internet service provider).
[0142] It should be noted that although several modules of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules described above can be embodied in one module. Conversely, the features and functions of one module described above can be further divided into multiple modules to be embodied.
[0143] In addition, although the operations of the disclosed method are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.
[0144] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk computer storage media, CD-ROM, optical computer storage media, etc.) containing computer-usable program codes.
[0145] The present disclosure is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0146] These computer program instructions may also be stored in a computer-readable computer storage medium that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable computer storage medium produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0147] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0148] Obviously, those skilled in the art can make various changes and modifications to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is also intended to include these modifications and variations.
Claims
1. A road area calibration method, characterized in that: The method comprises: For any frame of video image within a specified time length in the road video, a preset target detection neural network model is used to perform vehicle target detection on the video image to obtain the detection position of each target vehicle; For any target vehicle, the detection position of the target vehicle is matched with the predicted positions of each target vehicle respectively by using the Hungarian algorithm to determine the predicted position corresponding to the detection position of the target vehicle; wherein the predicted position of each target vehicle is obtained by predicting the detection position of each target vehicle in the previous frame of video image using the Kalman filter algorithm; if it is determined that the number of predicted positions corresponding to the detection position of the target vehicle is multiple, the predicted position with the highest priority is determined based on the cascade matching parameters corresponding to each predicted position, and the identification of the target vehicle in the previous frame of video image corresponding to the predicted position with the highest priority is determined as the identification of the target vehicle, wherein the cascade matching parameters are used to indicate the priority of the predicted position, and the identification of the same target vehicle in different video images is the same; For any target vehicle, the movement trajectory of the target vehicle on the road is determined by the identification and detection position of the target vehicle in each frame of the video image within the specified time period; Divide the motion trajectory of each target vehicle into a specified number of sub-motion trajectories, wherein the lengths of the sub-motion trajectories in the same motion trajectory are equal; Clustering each sub-motion trajectory using a preset clustering algorithm to obtain multiple motion trajectory clustering areas; For any motion trajectory clustering area, if it is determined that the number of each sub-motion trajectory in the motion trajectory clustering area is greater than a specified threshold, the motion trajectory clustering area is determined to be a vehicle dense area on the road.
2. The method according to claim 1, characterized in that The expansion rate of at least one target convolution in the backbone network of the target detection neural network model is a hole convolution with a set value, and the target convolution is a convolution of a specified size.
3. The method according to claim 1, characterized in that If it is determined that the number of predicted positions corresponding to the detected position of the target vehicle is one, the method further includes: The identification of the target vehicle in the previous frame of video image corresponding to the predicted position is determined as the identification of the target vehicle.
4. The method according to claim 1, characterized in that: For any target vehicle, the detection position of the target vehicle is matched with the predicted position of each target vehicle by using the Hungarian algorithm, and before determining the predicted position corresponding to the detection position of the target vehicle, the method further includes: For each target vehicle having a plurality of detection positions, the detection positions of each target vehicle are screened based on a preset algorithm to obtain the screened detection positions of each target vehicle.
5. The method according to claim 4, characterized in that The method of screening the detection positions of each target vehicle based on a preset algorithm to obtain the screened detection positions of each target vehicle includes: After the detection position of each target vehicle is screened by using a non-maximum suppression algorithm, the detection position of each target vehicle is screened again by the confidence of the detection position of each target vehicle to obtain the screened detection position of each target vehicle; wherein the confidence of the detection position of each target vehicle is obtained based on a preset target detection neural network model.
6. An electronic device, characterized in that: The device comprises a storage unit and a processor, wherein: The storage unit is configured to store the road video; The processor is configured to: For any frame of video image within a specified time length in the road video, a preset target detection neural network model is used to perform vehicle target detection on the video image to obtain the detection position of each target vehicle; For any target vehicle, the detection position of the target vehicle is matched with the predicted positions of each target vehicle respectively by using the Hungarian algorithm to determine the predicted position corresponding to the detection position of the target vehicle; wherein the predicted position of each target vehicle is obtained by predicting the detection position of each target vehicle in the previous frame of video image using the Kalman filter algorithm; if it is determined that the number of predicted positions corresponding to the detection position of the target vehicle is multiple, the predicted position with the highest priority is determined based on the cascade matching parameters corresponding to each predicted position, and the identification of the target vehicle in the previous frame of video image corresponding to the predicted position with the highest priority is determined as the identification of the target vehicle, wherein the cascade matching parameters are used to indicate the priority of the predicted position, and the identification of the same target vehicle in different video images is the same; For any target vehicle, the movement trajectory of the target vehicle on the road is determined by the identification and detection position of the target vehicle in each frame of the video image within the specified time period; Divide the motion trajectory of each target vehicle into a specified number of sub-motion trajectories, wherein the lengths of the sub-motion trajectories in the same motion trajectory are equal; Clustering each sub-motion trajectory using a preset clustering algorithm to obtain multiple motion trajectory clustering areas; For any motion trajectory clustering area, if it is determined that the number of each sub-motion trajectory in the motion trajectory clustering area is greater than a specified threshold, the motion trajectory clustering area is determined to be a vehicle dense area on the road.
7. The electronic device according to claim 6, characterized in that: The expansion rate of at least one target convolution in the backbone network of the target detection neural network model is a hole convolution with a set value, and the target convolution is a convolution of a specified size.
8. The electronic device according to claim 6, characterized in that: If it is determined that the number of predicted positions corresponding to the detected position of the target vehicle is one, the processor is further configured to: The identification of the target vehicle in the previous frame of video image corresponding to the predicted position is determined as the identification of the target vehicle.