Motorcade traffic monitoring method and device based on multi-modal data fusion

Abstract

The invention discloses a motorcade traffic monitoring method and device based on multi-modal data fusion, and the method comprises the steps: obtaining guarantee line information comprising a guarantee line coordinate sequence which comprises the coordinates of all inflection points on a guarantee line; therefore, the monitoring resource sequence corresponding to the guaranteed line is generated based on the guaranteed line coordinate sequence, namely, the road camera corresponding to each inflection point on the guaranteed line is determined; based on this, in actual use, a real-time monitoring video corresponding to the current position of the target motorcade can be obtained according to the position information of the target motorcade and based on each road camera in the monitoring resource sequence; and finally, the real-time monitoring video of the target motorcade at the current position is sent to a scheduling end, so that traffic monitoring of the target motorcade can be completed. By means of the design, compared with a traditional technology, the use cost is greatly reduced, and the use flexibility is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of traffic management informationization, and specifically relates to a vehicle fleet traffic monitoring method and device based on multimodal data fusion. Background Art

[0002] In order to ensure the traffic security of racing teams and important figures in major events, the traffic management department needs to monitor the location of the security vehicles and the surrounding environment in real time. At present, the main method used in the industry is to build a special "video following system", that is, to use the built-in visual recognition algorithm of the front-end facilities to realize automatic judgment and following of the target, and to relay the video between different cameras; or to use the vehicle-mounted unit to communicate with the drone through V2X to realize drone-based video following.

[0003] However, the above-mentioned existing technologies have the following deficiencies: (1) The video following method requires the construction of a large number of front-end vehicle video recognition terminals, which has high construction costs. In the scenario where multiple routes need to be protected at the same time, the economic efficiency is not good. At the same time, once the protection line is adjusted due to various reasons, it involves complicated and repetitive construction work such as equipment migration; (2) Although the drone solution can overcome the problem of line flexibility, it is still necessary to purchase multiple drone devices for the situation where multiple routes are protected at the same time, and the economic efficiency is still not good. In addition, the drone solution relies on the vehicle-side V2X terminal and needs to be specially configured on the vehicle. The implementation process is cumbersome, coordination is difficult, and the cost is high. Therefore, based on the above-mentioned deficiencies, how to provide a flexible and low-cost fleet traffic monitoring method has become an urgent problem to be solved. Summary of the invention

[0004] The purpose of the present invention is to provide a fleet traffic monitoring method and device based on multimodal data fusion, so as to solve the problems of high cost and poor flexibility in the prior art.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] In a first aspect, a vehicle fleet traffic monitoring method based on multimodal data fusion is provided, comprising:

[0007] Acquire the guarantee route information of the target fleet, wherein the guarantee route information includes a guarantee route coordinate sequence, each route coordinate in the guarantee route coordinate sequence is used to represent a turning point position in the guarantee route, and each route coordinate is sorted in order from the starting point to the end point of the guarantee route;

[0008] Generate a monitoring resource sequence corresponding to the protection route according to the protection route coordinate sequence, wherein the monitoring resource sequence includes a plurality of road cameras, the monitoring range of the plurality of road cameras covers all inflection points in the protection route, and each inflection point corresponds to a road camera;

[0009] Acquire the location information of the target fleet, and generate a real-time monitoring video of the target fleet based on the location information and the monitoring images of each road camera in the monitoring resource sequence;

[0010] The real-time monitoring video is sent to the dispatching end to complete the traffic monitoring of the target fleet after sending.

[0011] Based on the above disclosed content, the present invention first obtains the protection line information including the protection line coordinate sequence, wherein the protection line coordinate sequence includes the coordinates of each turning point (i.e., turning point) on the protection line; thus, the present invention generates a monitoring resource sequence corresponding to the protection line based on the protection line coordinate sequence, that is, determines the road camera corresponding to each turning point on the protection line; based on this, in actual use, the real-time monitoring video corresponding to the current position of the target fleet can be obtained according to the location information of the target fleet and based on each road camera in the monitoring resource sequence; finally, the real-time monitoring video of the target fleet at the current position is sent to the dispatching end, so that the traffic monitoring of the target fleet can be completed.

[0012] Through the above design, the present invention can complete the traffic monitoring of the fleet's route by utilizing the existing video perception sources around the route; thus, compared with the traditional technology, there is no need to additionally build video recognition terminals, use drones, and configure dedicated V2X equipment. Therefore, it not only greatly reduces the cost of use, but also improves the flexibility of use; therefore, the present invention is very suitable for large-scale application and promotion in the field of traffic security.

[0013] In a possible design, generating a monitoring resource sequence corresponding to the protection line according to the protection line coordinate sequence includes:

[0014] Obtain a GIS map of the area where the security line is located;

[0015] Determine a monitoring buffer area of ​​the protection line according to the protection line and the GIS map, wherein the monitoring buffer area covers the protection line;

[0016] Acquire all road cameras in the monitoring buffer area, and sort all acquired road cameras in a preset order, so as to generate a monitoring resource sequence corresponding to the security line after sorting, wherein the preset order is the arrangement order of each line coordinate in the security line coordinate sequence.

[0017] In a possible design, the real-time monitoring video of the target fleet includes monitoring videos at different times, wherein the real-time monitoring video of the target fleet is generated based on the location information and the monitoring screens of each road camera in the monitoring resource sequence, including:

[0018] Based on the position information, determining the position of the leading vehicle in the target vehicle fleet;

[0019] According to the position of the lead vehicle, multiple road cameras located on the guarantee route and in front of the lead vehicle are selected from the monitoring resource sequence, and a camera acquisition sequence is formed using the selected multiple road cameras, wherein the road cameras in the camera acquisition sequence are arranged in descending order according to the distance between them and the end point of the guarantee route;

[0020] Acquire a monitoring picture of a first road camera from the camera call sequence, and generate a monitoring video of the target convoy at the current moment based on the monitoring picture of the first road camera;

[0021] When the target convoy passes the first road camera, removing the first road camera from the camera acquisition sequence to obtain an updated camera acquisition sequence;

[0022] The camera retrieval sequence is replaced with the updated camera retrieval sequence, and the monitoring image of the first road camera is obtained from the camera retrieval sequence again until the target convoy reaches the end of the security route, thereby obtaining the monitoring video of the target convoy at different times.

[0023] In a possible design, the guarantee route information further includes: fleet shift information, wherein the method further includes:

[0024] Obtain speed information of the target convoy;

[0025] According to the position information, the speed information and the fleet shift information, the departure and arrival of the target fleet are monitored to obtain a shift monitoring result;

[0026] The shift monitoring result is sent to the dispatching end to complete the departure shift monitoring of the target fleet.

[0027] In a possible design, the fleet shift information includes the fleet departure time and the fleet arrival time, wherein the position information of the target fleet includes the first position, the second position and the running position of the lead vehicle in the target fleet, and the first position is the position of the lead vehicle when it arrives at the fleet departure time, the second position is the position of the lead vehicle when it arrives at the fleet arrival time, and the running position is the real-time position of the lead vehicle after the preset time of departure;

[0028] According to the position information, the speed information and the fleet shift information, the departure and arrival of the target fleet are monitored to obtain the shift monitoring result, including:

[0029] Based on the first position of the lead vehicle and the starting position of the guarantee route, the departure monitoring of the target vehicle fleet is performed to obtain the departure monitoring result of the target vehicle fleet;

[0030] According to the second position of the lead vehicle and the terminal position of the guarantee route, the target vehicle fleet is monitored for arrival, and an arrival monitoring result of the target vehicle fleet is obtained;

[0031] According to the speed information and the real-time position of the head vehicle in the target vehicle fleet, the target vehicle fleet is subjected to arrival prediction monitoring to obtain a prediction monitoring result;

[0032] The shift monitoring result is composed of the departure monitoring result, the arrival monitoring result and the forecast monitoring result.

[0033] In a possible design, the speed information is an average speed of the target convoy, wherein the target convoy is monitored for departure based on the first position of the lead vehicle and the starting position of the guarantee line, and the departure monitoring result of the target convoy is obtained, including:

[0034] Calculating the distance between the first position of the lead vehicle and the starting position to obtain a first distance;

[0035] Determining whether the first distance is less than a preset distance;

[0036] If yes, then after a specified period of time, the position of the lead vehicle is obtained again to obtain an updated first position;

[0037] Determining whether the distance between the updated first position and the starting position is less than the preset distance;

[0038] If yes, then the departure monitoring result of the target fleet is generated as delayed departure;

[0039] Accordingly, based on the speed information and the real-time position of the head vehicle in the target vehicle fleet, the target vehicle fleet is subjected to arrival prediction monitoring to obtain a prediction monitoring result, which includes:

[0040] Acquiring traffic operation status data of the guaranteed route;

[0041] Calculate the remaining mileage of the fleet based on the real-time position and the destination position;

[0042] Calculate the travel time of the target fleet to the destination location based on the remaining mileage of the fleet, the average speed and the traffic operation status data;

[0043] Based on the travel time, determining an estimated arrival time of the fleet;

[0044] If the estimated arrival time of the convoy is earlier than the arrival time of the convoy, the predicted monitoring result generated is that the convoy arrives in advance; if the estimated arrival time of the convoy is later than the arrival time of the convoy, the predicted monitoring result generated is that the convoy arrives late.

[0045] In one possible design, the method further includes:

[0046] Determine whether there is a conflicting intersection on the security route, wherein the conflicting intersection is used to represent the intersection position of the security route of the target fleet and the security routes of other designated fleets;

[0047] If yes, obtaining a second distance between the target convoy and the conflicting intersection, and a third distance between the remaining designated convoys and the conflicting intersection;

[0048] Determining whether the second distance and the third distance are both less than a minimum conflict distance;

[0049] If so, a conflict warning is generated and sent to the scheduling end.

[0050] In one possible design, the method further includes:

[0051] Acquiring traffic data of the guaranteed route;

[0052] Based on the traffic data, determining whether there is traffic congestion on the guaranteed route;

[0053] If yes, obtain a GIS map of the area where the guaranteed route is located, and generate an alternative route based on the GIS map;

[0054] The alternative routes are sent to the dispatching end to complete the optimization processing of the travel route of the target fleet.

[0055] In a second aspect, a vehicle fleet traffic monitoring device based on multimodal data fusion is provided, comprising:

[0056] An acquisition unit, configured to acquire the guarantee route information of the target vehicle fleet, wherein the guarantee route information includes a guarantee route coordinate sequence, each route coordinate in the guarantee route coordinate sequence is respectively used to represent a turning point position in the guarantee route, and each route coordinate is sorted in order from the starting point to the end point of the guarantee route;

[0057] A monitoring resource construction unit, configured to generate a monitoring resource sequence corresponding to the protection route according to the protection route coordinate sequence, wherein the monitoring resource sequence includes a plurality of road cameras, the monitoring range of the plurality of road cameras covers all inflection points in the protection route, and each inflection point corresponds to a road camera;

[0058] A monitoring video generation unit, used to obtain the location information of the target vehicle fleet, and generate a real-time monitoring video of the target vehicle fleet based on the location information and the monitoring images of each road camera in the monitoring resource sequence;

[0059] The sending unit is used to send the real-time monitoring video to the dispatching end, so as to complete the traffic monitoring of the target fleet after sending.

[0060] In a third aspect, another fleet traffic monitoring device based on multimodal data fusion is provided. Taking the device as an electronic device as an example, it includes a memory, a processor and a transceiver that are communicatively connected in sequence, wherein the memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the fleet traffic monitoring method based on multimodal data fusion as described in the first aspect or any possible design of the first aspect.

[0061] In a fourth aspect, a storage medium is provided, on which instructions are stored. When the instructions are executed on a computer, the fleet traffic monitoring method based on multimodal data fusion as described in the first aspect or any possible design of the first aspect is executed.

[0062] In a fifth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, causes the computer to execute the fleet traffic monitoring method based on multimodal data fusion as described in the first aspect or any possible design of the first aspect.

[0063] Beneficial effects:

[0064] (1) The present invention can complete traffic monitoring of the fleet's route by utilizing the existing video sensing sources around the route. Compared with the traditional technology, there is no need to build additional video recognition terminals, use drones, or configure dedicated V2X equipment. Therefore, it not only greatly reduces the cost of use, but also improves the flexibility of use. Therefore, the present invention is very suitable for large-scale application and promotion in the field of traffic security.

[0065] (2) The present invention can also monitor the departure and arrival of the fleet based on the departure and arrival time of the fleet and in combination with the position and speed of the fleet; in this way, it can monitor whether the fleet departs and arrives on time, thereby realizing the fleet's shift monitoring; at the same time, the present invention can also predict the time when the fleet arrives at the destination based on the real-time position, average speed and traffic operation status data of the route of the fleet; based on this, when it is predicted that the fleet may arrive late, an early warning prompt can be issued, so that the dispatcher can take corresponding measures to ensure that the fleet arrives on time.

[0066] (3) The present invention can provide conflict warning for the security convoy when multiple security tasks are being carried out simultaneously, that is, when there is an intersection between two routes and the convoy enters the preset range of the intersection at the same time, a conflict warning is issued; thereby, coordinated command of multiple tasks is achieved, ensuring the reliable implementation of the security tasks.

[0067] (4) The present invention can also monitor the congestion ahead of the convoy in advance based on the traffic data of the guaranteed route and in combination with the GIS map of the area where it is located, and automatically provide route optimization suggestions or alternative routes; in this way, it can assist command and dispatch decision-making, thereby ensuring the smooth passage of the convoy. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 A schematic flow chart of the steps of a vehicle fleet traffic monitoring method based on multimodal data fusion provided by an embodiment of the present invention;

[0069] Figure 2 A schematic diagram of the structure of a vehicle fleet traffic monitoring device based on multimodal data fusion provided by an embodiment of the present invention;

[0070] Figure 3 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0071] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in combination with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the structure of the drawings is only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0072] It should be understood that although the terms first, second, etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are only used to distinguish one unit from another unit. For example, a first unit can be referred to as a second unit, and similarly, a second unit can be referred to as a first unit without departing from the scope of the exemplary embodiments of the present invention.

[0073] It should be understood that the term "and / or" that may appear in this article is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can represent three situations: A exists alone, B exists alone, and A and B exist at the same time. The term " / and" that may appear in this article describes another type of association object relationship, indicating that two relationships may exist. For example, A / and B can represent two situations: A exists alone, and A and B exist alone. In addition, the character " / " that may appear in this article generally indicates that the previous and next associated objects are in an "or" relationship.

[0074] Example:

[0075] See also Figure 1 As shown, the fleet traffic monitoring method based on multimodal data fusion provided in this embodiment obtains the coordinates of each turning point on the guarantee line, and then, based on the coordinates of each turning point and in combination with the existing video perception sources (i.e., road cameras) around the line, establishes a monitoring resource sequence corresponding to the guarantee line; then, based on the location information of the fleet and according to the images monitored by each road camera in the monitoring resource sequence, the traffic monitoring of the fleet's route can be completed; thus, this method does not require the additional construction of video recognition terminals, the use of drones, and the configuration of dedicated V2X equipment, therefore, not only greatly reduces the cost of use, but also improves the flexibility of use, so that it is very suitable for large-scale application and promotion in the field of traffic security; among them, for example, this method can be but not limited to running on the scheduling server side, it can be understood that the aforementioned execution subject does not constitute a limitation on the embodiments of the present application, and accordingly, the operation steps of this method can be but not limited to the following steps S1 to S4.

[0076] S1. Obtain the guarantee route information of the target fleet, wherein the guarantee route information includes a guarantee route coordinate sequence, each route coordinate in the guarantee route coordinate sequence is respectively used to represent a turning point position in the guarantee route, and each route coordinate is sorted in order from the starting point to the end point of the guarantee route; in this embodiment, two methods for obtaining the guarantee route coordinate sequence are provided, the first method is: in response to a user click operation, to obtain all turning point positions (i.e., turning points, that is, turning points) of the guarantee route from the starting point to the end point on the local GIS map; then, based on all the turning point positions, to generate the guarantee route coordinate sequence; at the same time, the second method is: in response to a data input operation, to obtain the guarantee route coordinate sequence.

[0077] Furthermore, the guaranteed route information mentioned in the example may also include but is not limited to: fleet shift information, wherein the fleet shift information may include fleet departure time, fleet arrival time, departure shift, license plate number, driver name, telephone number and other basic information; in this way, combined with the aforementioned information, a complete guaranteed route information can be formed; then, all guaranteed route information is entered into the database, and it can be queried in the system database according to the departure time, departure location, etc.; of course, the aforementioned fleet departure time and fleet arrival time are both planned times.

[0078] In this way, after obtaining the security route information of the target fleet, a video resource sequence, that is, the following monitoring resource sequence, can be constructed, so that real-time monitoring of the target fleet during its movement can be completed based on the video resource sequence; wherein, the construction process of the monitoring resource sequence can be but is not limited to as shown in the following step S2.

[0079] S2. Generate a monitoring resource sequence corresponding to the security line according to the coordinate sequence of the security line, wherein the monitoring resource sequence includes a number of road cameras, the monitoring range of the several road cameras covers all turning point positions in the security line, and each turning point position corresponds to a road camera; in specific implementation, it is based on the road network cognitive basic data (this data refers to the geographic coordinates containing vector maps and all video resources), and quickly obtains the video perception source (i.e., road camera) around the security line on the local GIS map, and then, based on this, forms a monitoring resource sequence.

[0080] Optionally, the specific construction process of the aforementioned monitoring resource sequence may be but is not limited to the following steps S21 to S23.

[0081] S21. Obtain a GIS map of the area where the security line is located; in the present embodiment, the GIS map of the area where the security line is located refers to the aforementioned local GIS map, which contains the aforementioned road network recognition basic data; therefore, based on the GIS map, road cameras around the security line can be quickly screened out, and the process can be but is not limited to as shown in the following steps S22 and S23.

[0082] S22. According to the security line and the GIS map, the monitoring buffer area of ​​the security line is determined, wherein the monitoring buffer area covers the security line; in specific implementation, the specified distance is extended to both sides of the security line to form a strip area, and the area is the monitoring buffer area; optionally, the specified distance can be but is not limited to 30m, 50m or 100m, which can be specifically set according to actual use and is not limited to the aforementioned example.

[0083] In this way, after obtaining the monitoring buffer area corresponding to the protection line, the road cameras in the monitoring buffer area can be extracted, and based on this, a monitoring resource sequence can be constructed; wherein the construction process is shown in the following step S23.

[0084] S23. Acquire all the road cameras in the monitoring buffer area, and sort all the acquired road cameras in a preset order, so as to generate a monitoring resource sequence corresponding to the protection line after sorting, wherein the preset order is the arrangement order of the coordinates of each line in the protection line coordinate sequence; in specific application, it is equivalent to sorting the road cameras in the monitoring buffer area according to the position of each turning point, that is, sorting them in order from the starting point to the end point; at the same time, there may be a situation where there are multiple road cameras corresponding to one turning point, therefore, after the sorting is completed, it will be sent to the audit end so that the auditors at the audit end can review it to determine the best road camera corresponding to each turning point; then, the audit end uploads the reviewed sequence to the traffic dispatch server, at this time, the monitoring resource sequence corresponding to the protection line can be obtained.

[0085] In this embodiment, the example road cameras may include but are not limited to: all road test intelligent traffic video cameras (checkpoints, electronic police, video surveillance) gathered by the public security traffic management department and the gathered public security Skynet cameras.

[0086] In this way, through the aforementioned steps S21 to S23, the existing video perception sources around the security line can be used to construct a monitoring resource sequence corresponding to the security line; then, based on this, the movement of the target fleet can be monitored, and the process can be but not limited to as shown in the following step S3.

[0087] S3. Obtain the location information of the target fleet, and generate a real-time monitoring video of the target fleet based on the location information and the monitoring images of each road camera in the monitoring resource sequence; in specific applications, the location information may include but is not limited to the location of the lead vehicle of the target fleet, that is, the real-time location of the lead vehicle, so that the road camera in the monitoring resource sequence can be determined according to the real-time location of the lead vehicle, so as to monitor the fleet; at the same time, the location information may be obtained according to but is not limited to the GPS signal of the lead vehicle, and after obtaining, the GPS signal may be pre-processed first, such as performing data pre-processing such as timing correction, denoising, equalization, and standardization on the satellite positioning information in the GPS signal to ensure the accuracy of the data.

[0088] Furthermore, for example, the real-time monitoring video of the target fleet includes monitoring videos at different times, that is, monitoring videos at different locations. In this embodiment, the following steps S31 to S35 are used to obtain the aforementioned real-time monitoring video.

[0089] S31. Based on the location information, determine the position of the lead vehicle in the target convoy; in this embodiment, after obtaining the position of the lead vehicle according to the GPS signal of the lead vehicle, the road camera used to monitor the target convoy can be determined from the monitoring resource sequence according to the position of the lead vehicle, and the process can be but is not limited to as shown in the following step S32.

[0090] S32. According to the position of the lead vehicle, multiple road cameras located on the security route and in front of the lead vehicle are selected from the monitoring resource sequence, and a camera retrieval sequence is composed of the selected multiple road cameras, wherein each road camera in the camera retrieval sequence is arranged in descending order according to the distance from the end point of the security route; in specific implementation, it is equivalent to determining each road camera located in front of the target convoy (i.e., the camera between the lead vehicle and the end point) according to the position of the lead vehicle and the position of each road camera (which can be obtained according to the coordinates of the turning points corresponding to each road camera); then, the road cameras located in front of the lead vehicle are used to form a camera retrieval sequence; finally, the monitoring screen can be retrieved according to each road camera in the camera retrieval sequence, thereby obtaining the monitoring video of the target convoy at different times, and the process is shown in the following steps S33 to S35.

[0091] S33. Obtain the monitoring image of the first road camera from the camera retrieval sequence, and generate the monitoring video of the target fleet at the current moment based on the monitoring image of the first road camera; in this embodiment, the first road camera in the camera retrieval sequence is the camera closest to the position of the lead vehicle, so its corresponding monitoring image is the real-time video of the target fleet at the current position; therefore, the monitoring image of the first road camera can be directly sent to the dispatching end, thereby realizing traffic monitoring of the current position of the target fleet.

[0092] At the same time, as the target convoy moves, it is necessary to switch the camera, that is, to obtain the next road camera in the camera scheduling sequence, and upload the monitoring picture of the next road camera, so as to realize traffic monitoring when the target convoy moves to the next position; in this way, by continuously retrieving the monitoring picture in this way, the monitoring picture of the target convoy at different times can be obtained, thereby realizing video relay; wherein, the video relay process is shown in the following steps S34 and S35.

[0093] S34. After the target convoy passes the first road camera, the first road camera is removed from the camera retrieval sequence to obtain an updated camera retrieval sequence; in specific applications, for example, when the target convoy leaves the first road camera in the camera retrieval sequence 10 seconds later, the camera sequence is updated and the monitoring screen is re-retrieved; of course, the position of the last vehicle can be determined based on the GPS information of the last vehicle in the target convoy, and the time when it passes the first road camera is recorded, and then, after 10 seconds, the camera retrieval sequence is updated; in addition, the update interval of the aforementioned camera retrieval sequence can be specifically set according to actual use, and is not limited to the aforementioned example.

[0094] In this way, after completing the update of the camera scheduling sequence, the monitoring screen can be retrieved again to achieve uninterrupted monitoring of the target fleet, and the process is shown in the following step S35.

[0095] S35. Replace the camera retrieval sequence with the updated camera retrieval sequence, and re-acquire the monitoring image of the first road camera from the camera retrieval sequence, until the target convoy reaches the end point of the security route, and obtain the monitoring video of the target convoy at different times; in this embodiment, it is equivalent to moving the camera retrieval sequence forward one step, updating the road camera to be retrieved in the sequence, and then re-retrieving the monitoring image of the first road camera in the updated sequence; in this way, the existing cameras can be used to realize automatic video relay of the convoy.

[0096] Of course, each time the monitoring screen of a road camera is retrieved, it is equivalent to obtaining a real-time video of the target convoy at a position. Therefore, when the target convoy reaches the end of the security route, its monitoring video at different times can be obtained; in addition, for example, the monitoring screen of the front-view camera of the head vehicle in the target convoy can also be obtained and uploaded to the dispatching end in real time to realize real-time monitoring of the road condition of the head vehicle.

[0097] Based on the aforementioned steps S31 to S35, the real-time monitoring video of the target fleet can be obtained. Finally, the video is uploaded to complete the traffic monitoring of the target fleet. The process is shown in the following step S4.

[0098] S4. Send the real-time monitoring video to the dispatching end, so as to complete the traffic monitoring of the target fleet after sending; in this embodiment, it has been explained above that the real-time monitoring video actually includes the monitoring videos of the target fleet at different times. Therefore, by uploading each monitoring video in real time (that is, uploading it in real time after each monitoring video is obtained), real-time monitoring of the fleet can be achieved.

[0099] Therefore, through the fleet traffic monitoring method based on multimodal data fusion described in detail in the aforementioned steps S1 to S4, the present invention obtains the coordinates of each turning point on the protection line, and then, based on the coordinates of each turning point and in combination with the existing video perception sources around the line, establishes a monitoring resource sequence corresponding to the protection line; then, based on the location information of the fleet and according to the images monitored by each road camera in the monitoring resource sequence, the traffic monitoring of the fleet's route can be completed; therefore, the present invention does not need to additionally build a video recognition terminal, use a drone, and configure a dedicated V2X device, therefore, not only greatly reduces the cost of use, but also improves the flexibility of use, so it is very suitable for large-scale application and promotion in the field of traffic security.

[0100] In a possible design, the second aspect of this embodiment provides a further fleet monitoring solution based on the first aspect of the embodiment, that is, utilizing the vehicle schedule information and fleet location information in the guarantee route information to achieve departure and arrival monitoring of the target fleet; wherein, the departure and arrival monitoring process can be, but is not limited to, as shown in the following steps S5 to S7.

[0101] S5. Obtain the speed information of the target fleet. In this embodiment, the speed information is taken as the average speed of the target fleet. After obtaining the average speed of the target fleet, the departure and arrival monitoring of the target fleet can be performed in combination with the aforementioned fleet shift information and location information. The process is shown in the following step S6.

[0102] S6. Monitor the departure and arrival of the target fleet based on the position information, the speed information and the fleet shift information to obtain the shift monitoring results; in the specific implementation, the aforementioned position information may also include, but is not limited to: the first position, the second position and the running position of the lead vehicle in the target fleet, and the first position is the position of the lead vehicle when it arrives at the departure time of the fleet, the second position is the position of the lead vehicle when it arrives at the arrival time of the fleet, and the running position is the real-time position of the lead vehicle after a preset time (such as 5 minutes) from departure; in this way, it is equivalent to monitoring whether the vehicle departs on time, warning of vehicle delays, and predicting whether the vehicle arrives early or late based on the aforementioned data, and making corresponding warnings according to the monitoring results.

[0103] The above-mentioned fleet departure and arrival monitoring process may be but is not limited to the following steps S61 to S64.

[0104] S61. Based on the first position of the lead vehicle and the starting position of the support line, the target vehicle fleet is monitored for departure, and the departure monitoring result of the target vehicle fleet is obtained; in this embodiment, departure monitoring is actually to monitor whether the vehicle fleet departs on time, and its specific implementation process is shown in the following steps S61a~S61e.

[0105] S61 a. Calculate the distance between the first position of the lead vehicle and the starting position to obtain the first distance; in this embodiment, it is equivalent to calculating the distance between the actual position of the lead vehicle and the departure point after the departure time of the fleet is reached, and then, judging whether the lead vehicle has left the departure point based on the calculated distance, and the judgment process is shown in the following step S61 b.

[0106] S61 b. Determine whether the first distance is less than a preset distance; in specific applications, the preset distance can be, but is not limited to, set to 100m (of course, the distance can be set specifically according to actual use), that is, if the distance between the actual position of the lead vehicle and the departure point is greater than or equal to 100, it means that the lead vehicle has left the departure point; on the contrary, if it is less than 100, it means that the lead vehicle has not yet left the departure point. At this time, continuous judgment is required, and the process is shown in the following steps S61 c to S61e.

[0107] S61c. If so, then after a specified period of time, the position of the lead vehicle is obtained again to obtain an updated first position; in specific implementation, the position of the lead vehicle can be obtained again after 1 minute, at which time, the distance between the current position of the lead vehicle and the departure point is continued to be calculated, and then a judgment is made again, and the process is shown in the following steps S61d and S61e.

[0108] S61d. Determine whether the distance between the updated first position and the starting position is less than the preset distance; in this embodiment, if after a specified period of time, the distance between the lead vehicle and the departure point is greater than or equal to 100, it means that the fleet has departed; otherwise, if it is still less than 100, it means that the fleet is delayed in departure. At this time, a departure monitoring result can be generated as delayed departure, and the process is shown in the following step S61e.

[0109] S61d. If yes, the departure monitoring result of the target fleet is generated as delayed departure; in this embodiment, when the departure monitoring result is delayed departure, a departure delay warning can be generated to the dispatching end so that the dispatching personnel can contact the driver to understand the situation and take corresponding countermeasures.

[0110] Thus, through the aforementioned steps S61a to S61e, the departure monitoring of the target fleet can be completed; then, the arrival monitoring can be performed, and the process is shown in the following step S62.

[0111] S62. According to the second position of the lead vehicle and the terminal position of the security line, the target vehicle fleet is monitored for arrival, and the arrival monitoring result of the target vehicle fleet is obtained; in specific applications, the principle of vehicle fleet arrival monitoring is the same as that of vehicle departure monitoring, that is, the position of the lead vehicle at the time of the vehicle fleet arrival and the distance from the terminal are calculated, that is, the distance between the second position and the terminal position is calculated, and then it is determined whether it is greater than the preset distance (still 100m for example). If so, it means that the lead vehicle has not yet arrived at the terminal. At this time, continuous judgment is required, that is: after a specified period of time, the position of the lead vehicle is obtained again, and then the distance between it and the terminal is calculated, and it is judged again. If it is still greater than the preset distance, the arrival monitoring result is generated as vehicle fleet lateness; at the same time, a vehicle fleet lateness warning can also be generated and sent to the dispatch end.

[0112] In addition, in this embodiment, when there are multiple support tasks (that is, there are multiple vehicle shifts), at this time, the shifts and number of delayed vehicles and the shifts and number of late arriving vehicles can be counted based on the aforementioned departure and arrival monitoring results to facilitate subsequent data analysis.

[0113] After completing the departure and arrival monitoring of the fleet, this embodiment can also predict the delayed or early arrival of the fleet, and the process is shown in the following step S63.

[0114] S63. Based on the speed information and the real-time position of the head vehicle in the target fleet, the target fleet is subjected to arrival prediction monitoring to obtain prediction monitoring results. In this embodiment, the following steps S63a to S63e may be used as an example but are not limited to achieve arrival prediction monitoring of the fleet.

[0115] S63a. Obtain the traffic operation status data of the security line; in this embodiment, the traffic operation status data may be, but is not limited to, the congestion data of the security line, such as a 1000m congestion at position xx of the security line, and the estimated passing time is xx minutes, etc.; of course, the aforementioned traffic operation status data may be obtained based on a real-time local GIS map, and after obtaining the aforementioned traffic operation status data, data preprocessing may also be performed to ensure the accuracy of the data.

[0116] After obtaining the traffic operation status data, the remaining mileage can be calculated based on the real-time position of the lead vehicle and the terminal position, so that the estimated arrival time of the fleet can be calculated based on the remaining mileage; wherein the above process can be but is not limited to as shown in the following steps S63b to S63d.

[0117] S63b. Based on the real-time position and the terminal position, the remaining mileage of the fleet is calculated; in this embodiment, after the remaining mileage is obtained, the time taken for the fleet to travel to the terminal can be calculated in combination with the aforementioned traffic operation status data and average speed, and the process is shown in the following step S63c.

[0118] S63c. Calculate the driving time of the target fleet to the terminal location based on the remaining mileage of the fleet, the average speed and the traffic operation status data; in specific applications, first subtract the congested distance in the traffic operation status data from the remaining mileage of the fleet to obtain the uncongested distance; then, calculate the theoretical driving time using the uncongested distance and the average speed; finally, add the estimated travel time of the congested section in the traffic operation status data to obtain the driving time of the target fleet to the terminal location; of course, the driving time changes in real time according to the changes in the traffic operation status data, and therefore, can be updated at intervals.

[0119] In this way, after obtaining the travel time of the target convoy to the destination, the estimated arrival time of the convoy can be determined based on this, and the process is shown in the following step S63d.

[0120] S63d. Based on the driving time, the estimated arrival time of the convoy is determined; in this embodiment, after the estimated arrival time of the convoy is obtained, it is compared with the aforementioned proposed arrival time of the convoy to determine whether the target convoy arrives ahead of schedule or delayed; wherein, the comparison process is shown in the following step S63e.

[0121] S63e. If the estimated arrival time of the convoy is earlier than the arrival time of the convoy, a prediction monitoring result is generated as the convoy's early arrival; if the estimated arrival time of the convoy is later than the arrival time of the convoy, a prediction monitoring result is generated as the convoy's delayed arrival; in specific implementation, a time threshold can be set to perform an early arrival warning. For example, when the estimated arrival time of the convoy is earlier than the arrival time of the convoy, and the time difference between the two reaches the time threshold (such as 5 minutes, that is, the convoy is predicted to arrive 5 minutes in advance), at this time, a convoy early arrival warning can be generated and sent to the dispatching end, so that the dispatching personnel can promptly notify the staff at the terminal to make corresponding preparations; at the same time, if the estimated arrival time of the convoy is later than the arrival time of the convoy, a convoy arrival delay warning can be directly generated and sent to the dispatching end, so that the dispatching personnel can command the convoy drivers to speed up, thereby ensuring that the convoy arrives on time.

[0122] In this way, through the aforementioned steps S63a to S63e, the early or delayed arrival prediction of the fleet can be completed, and then, the aforementioned departure and arrival monitoring can be combined to generate the fleet's shift monitoring results, and the process is shown in the following step S64.

[0123] S64. Utilize the departure monitoring result, the arrival monitoring result and the forecast monitoring result to form the shift monitoring result.

[0124] Through the aforementioned steps S61 to S64, the departure and arrival monitoring of the fleet can be completed, and then the final monitoring result can be sent to the dispatching end to achieve corresponding early warning, and the process is shown in the following step S7.

[0125] S7. Send the shift monitoring result to the dispatching end to complete the departure shift monitoring of the target fleet; in this embodiment, the sending of different departure, arrival monitoring and prediction monitoring results and the corresponding warnings have been explained above and will not be repeated here.

[0126] Based on the aforementioned steps S5 to S7, it is possible to monitor whether the fleet departs and arrives on time, and to implement early arrival and delayed arrival warnings for the fleet; in this way, real-time monitoring of the fleet's shifts can be achieved, thereby ensuring that the fleet arrives at the destination on time.

[0127] Furthermore, in this embodiment, a conflict warning of the convoy can also be performed, and the process is as follows:

[0128] S8. Determine whether there is a conflict intersection on the security route, wherein the conflict intersection is used to represent the intersection position of the security route of the target fleet and the security routes of other designated fleets; in this embodiment, if the current security route intersects with other security routes, it can be determined that there is a conflict intersection; at this time, it is necessary to issue a conflict warning when the two fleets enter the intersection at the same time, and the process is shown in the following steps S9 to S11.

[0129] S9. If yes, then obtain the second distance between the target convoy and the conflict intersection, and the third distance between the remaining designated convoys and the conflict intersection; in specific implementation, the corresponding position is obtained based on the GPS signal of the head vehicle of each convoy, and then the distance between the head vehicle and the conflict intersection is calculated; then, it is necessary to determine whether the aforementioned second distance and third distance are both less than the minimum conflict distance, so as to perform conflict warning based on the judgment result, and the process is shown in the following steps S10 and S11.

[0130] S10. Determine whether the second distance and the third distance are both smaller than the minimum conflict distance. In specific implementation, the minimum conflict distance may be, but is not limited to, set to 1 km, i.e., when the target convoy and the other designated convoys simultaneously enter the range of 1 km from the conflict intersection, a conflict warning needs to be generated; otherwise, no conflict warning is required. The conflict warning process is shown in the following step S11.

[0131] S11. If yes, a conflict warning is generated and sent to the dispatching end. In this embodiment, when the dispatching end receives the conflict warning, it can avoid the risk of fleet conflict by notifying the fleet drivers to speed up or slow down.

[0132] In this way, through the aforementioned steps S9 to S11, a convoy conflict warning can be achieved, thereby ensuring the driving safety of the convoy.

[0133] In addition, in this embodiment, the route can also be optimized according to the road conditions ahead of the guaranteed route, and the process can be but not limited to the following steps S12 to S15.

[0134] S12. Obtain the traffic data of the guaranteed route; in specific implementation, the traffic data may be, but is not limited to, road condition data, and of course, is obtained based on a real-time local GIS map; after obtaining the traffic data, data preprocessing may be performed, such as denoising, standardization, etc.; then, it may be determined whether there is traffic congestion on the guaranteed route, so as to optimize the route according to the congestion condition, and the process is shown in the following steps S13 and S14.

[0135] S13. Based on the traffic data, determine whether there is traffic congestion on the guaranteed route; in this embodiment, when the traffic volume of a certain section of the road condition data is greater than a preset value, it can be determined that there is traffic congestion on the guaranteed route. At this time, route optimization can be performed, that is, generating alternative routes, and the process is shown in the following step S14.

[0136] S14. If so, obtain a GIS map of the area where the guaranteed route is located, and generate an alternative route based on the GIS map; in the specific implementation, it is equivalent to automatically planning a route with the shortest travel time based on the GIS map and according to the current location of the target fleet and the terminal location of the guaranteed route, so as to serve as an alternative route.

[0137] After the alternative routes are obtained, they can be sent to the dispatching end to optimize the travel route, and the process is shown in the following step S15.

[0138] S15. Send the alternative route to the dispatching end to complete the optimization processing of the travel route of the target fleet; in this embodiment, after receiving the alternative route, the dispatching end displays it, and after review by the dispatching personnel, it can be sent to the target fleet, so that the target fleet can travel along the alternative route to reach the destination on time.

[0139] In this way, through the aforementioned steps S12 to S15, the convoy's travel route can be optimized, thereby ensuring the convoy's smooth travel and punctual arrival.

[0140] In addition, this embodiment provides an application example of this method, namely:

[0141] During the xx event, more than 3,000 vehicles and 53 convoys for the opening ceremony needed to assemble at a stadium from assembly points in five directions across the city in less than four hours. The application ensured the accurate arrival of all vehicles throughout the process, fully verifying the effect of innovative multimodal data fusion, and achieving zero delays for the fleet under extremely complex circumstances with multiple passenger groups running concurrently.

[0142] During the event, the video relay function achieved a 10-meter following accuracy, assisting command and dispatch personnel to promptly discover and deal with 8 sudden traffic incidents involving vehicles on the line; the vehicle operation status recognition and early warning function achieved a time calculation error within 10 seconds, accurately predicted 150 early / delayed departures of event vehicles, 70 event vehicle operation delays, 28 convoy conflicts and 17 convoy tailgating situations, gave 3 route optimization suggestions, and suggested the selection of 2 alternative routes, significantly improving the timeliness, accuracy and intelligence of traffic security command and dispatch; combined with real-time command and dispatch measures, it ultimately ensured "zero error, zero delay, zero accident and zero complaint" for event traffic security.

[0143] like Figure 2 As shown, the third aspect of this embodiment provides a hardware device for implementing the vehicle fleet traffic monitoring method based on multimodal data fusion described in the first aspect of the embodiment, including:

[0144] An acquisition unit is used to acquire the guarantee route information of the target fleet, wherein the guarantee route information includes a guarantee route coordinate sequence, each route coordinate in the guarantee route coordinate sequence is used to represent a turning point position in the guarantee route, and each route coordinate is sorted in order from the starting point to the end point of the guarantee route.

[0145] A monitoring resource construction unit is used to generate a monitoring resource sequence corresponding to the security line according to the security line coordinate sequence, wherein the monitoring resource sequence includes a plurality of road cameras, the monitoring range of the plurality of road cameras covers all turning point positions in the security line, and each turning point position corresponds to a road camera.

[0146] The monitoring video generation unit is used to obtain the location information of the target vehicle fleet, and generate a real-time monitoring video of the target vehicle fleet based on the location information and the monitoring images of each road camera in the monitoring resource sequence.

[0147] The sending unit is used to send the real-time monitoring video to the dispatching end, so as to complete the traffic monitoring of the target fleet after sending.

[0148] The working process, working details and technical effects of the device provided in this embodiment can be found in the first aspect of the embodiment and will not be described in detail here.

[0149] like Figure 3 As shown, the fourth aspect of this embodiment provides another fleet traffic monitoring device based on multimodal data fusion. Taking the device as an electronic device as an example, it includes: a memory, a processor and a transceiver that are communicatively connected in sequence, wherein the memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the fleet traffic monitoring method based on multimodal data fusion as described in the first and second aspects of the embodiments.

[0150] For example, the memory may include, but is not limited to, random access memory (RAM), read-only memory (ROM), flash memory, first-in-first-out memory (FIFO) and / or first-in-last-out memory (FILO), etc. Specifically, the processor may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). At the same time, the processor may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state.

[0151] In some embodiments, the processor may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. For example, the processor may not be limited to a microprocessor of the STM32F105 series, a reduced instruction set computer (RISC) microprocessor, an X86 or other architecture processor, or a processor with an integrated embedded neural network processor (NPU); the transceiver may be, but is not limited to, a wireless fidelity (WIFI) wireless transceiver, a Bluetooth wireless transceiver, a general packet radio service technology (Genera l Packet Rad ioService, GPRS) wireless transceiver, a ZigBee protocol (a low-power LAN protocol based on the IEEE 802.15.4 standard, ZigBee) wireless transceiver, a 3G transceiver, a 4G transceiver and / or a 5G transceiver, etc. In addition, the device may also include, but is not limited to, a power module, a display screen and other necessary components.

[0152] The working process, working details and technical effects of the electronic device provided in this embodiment can be found in the first and second aspects of the embodiment, and will not be described in detail here.

[0153] A fifth aspect of the present embodiment provides a storage medium storing instructions for the fleet traffic monitoring method based on multimodal data fusion as described in the first and second aspects of the embodiment, that is, the storage medium stores instructions, and when the instructions are executed on a computer, the fleet traffic monitoring method based on multimodal data fusion as described in the first and second aspects of the embodiment is executed.

[0154] The storage medium refers to a carrier for storing data, which may include but is not limited to a floppy disk, a CD, a hard disk, a flash memory, a USB flash drive and / or a memory stick, etc. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0155] The working process, working details and technical effects of the storage medium provided in this embodiment can be found in the first and second aspects of the embodiments, and will not be described in detail here.

[0156] A sixth aspect of this embodiment provides a computer program product comprising instructions, which, when executed on a computer, causes the computer to execute the fleet traffic monitoring method based on multimodal data fusion as described in the first and second aspects of the embodiment, wherein the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0157] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A fleet traffic monitoring method based on multimodal data fusion, characterized in that: include: Acquire the guarantee route information of the target fleet, wherein the guarantee route information includes a guarantee route coordinate sequence, each route coordinate in the guarantee route coordinate sequence is used to represent a turning point position in the guarantee route, and each route coordinate is sorted in order from the starting point to the end point of the guarantee route; Generate a monitoring resource sequence corresponding to the protection route according to the protection route coordinate sequence, wherein the monitoring resource sequence includes a plurality of road cameras, the monitoring range of the plurality of road cameras covers all inflection points in the protection route, and each inflection point corresponds to a road camera; Acquire the location information of the target fleet, and generate a real-time monitoring video of the target fleet based on the location information and the monitoring images of each road camera in the monitoring resource sequence; The real-time monitoring video is sent to the dispatching end to complete the traffic monitoring of the target fleet after sending.

2. The method according to claim 1, characterized in that Generating a monitoring resource sequence corresponding to the security line according to the security line coordinate sequence, including: Obtain a GIS map of the area where the security line is located; Determine a monitoring buffer area of ​​the protection line according to the protection line and the GIS map, wherein the monitoring buffer area covers the protection line; Acquire all road cameras in the monitoring buffer area, and sort all acquired road cameras in a preset order, so as to generate a monitoring resource sequence corresponding to the security line after sorting, wherein the preset order is the arrangement order of each line coordinate in the security line coordinate sequence.

3. The method according to claim 1, characterized in that The real-time monitoring video of the target fleet includes monitoring videos at different times, wherein the real-time monitoring video of the target fleet is generated based on the location information and the monitoring screens of each road camera in the monitoring resource sequence, including: Based on the position information, determining the position of the leading vehicle in the target vehicle fleet; According to the position of the lead vehicle, multiple road cameras located on the guarantee route and in front of the lead vehicle are selected from the monitoring resource sequence, and a camera acquisition sequence is formed using the selected multiple road cameras, wherein the road cameras in the camera acquisition sequence are arranged in descending order according to the distance between them and the end point of the guarantee route; Acquire a monitoring picture of a first road camera from the camera call sequence, and generate a monitoring video of the target convoy at the current moment based on the monitoring picture of the first road camera; When the target convoy passes the first road camera, removing the first road camera from the camera acquisition sequence to obtain an updated camera acquisition sequence; The camera retrieval sequence is replaced with the updated camera retrieval sequence, and the monitoring image of the first road camera is obtained from the camera retrieval sequence again until the target convoy reaches the end of the security route, thereby obtaining the monitoring video of the target convoy at different times.

4. The method according to claim 1, characterized in that: The guarantee route information also includes: fleet shift information, wherein the method further includes: Obtain speed information of the target convoy; According to the position information, the speed information and the fleet shift information, the departure and arrival of the target fleet are monitored to obtain a shift monitoring result; The shift monitoring result is sent to the dispatching end to complete the departure shift monitoring of the target fleet.

5. The method according to claim 4, characterized in that The fleet shift information includes the fleet departure time and the fleet arrival time, wherein the position information of the target fleet includes the first position, the second position and the running position of the lead vehicle in the target fleet, and the first position is the position of the lead vehicle when it arrives at the fleet departure time, the second position is the position of the lead vehicle when it arrives at the fleet arrival time, and the running position is the real-time position of the lead vehicle after the preset time of departure; According to the position information, the speed information and the fleet shift information, the departure and arrival of the target fleet are monitored to obtain the shift monitoring result, including: Based on the first position of the lead vehicle and the starting position of the guarantee route, the departure monitoring of the target vehicle fleet is performed to obtain the departure monitoring result of the target vehicle fleet; According to the second position of the lead vehicle and the terminal position of the guarantee route, the target vehicle fleet is monitored for arrival, and an arrival monitoring result of the target vehicle fleet is obtained; According to the speed information and the real-time position of the head vehicle in the target vehicle fleet, the target vehicle fleet is subjected to arrival prediction monitoring to obtain a prediction monitoring result; The shift monitoring result is composed of the departure monitoring result, the arrival monitoring result and the forecast monitoring result.

6. The method according to claim 5, characterized in that The speed information is the average speed of the target convoy, wherein the target convoy is monitored for departure based on the first position of the lead vehicle and the starting position of the guarantee line, and the departure monitoring result of the target convoy is obtained, including: Calculating the distance between the first position of the lead vehicle and the starting position to obtain a first distance; Determining whether the first distance is less than a preset distance; If yes, then after a specified period of time, the position of the lead vehicle is obtained again to obtain an updated first position; Determining whether the distance between the updated first position and the starting position is less than the preset distance; If yes, then the departure monitoring result of the target fleet is generated as delayed departure; Accordingly, based on the speed information and the real-time position of the head vehicle in the target vehicle fleet, the target vehicle fleet is subjected to arrival prediction monitoring to obtain a prediction monitoring result, which includes: Acquiring traffic operation status data of the guaranteed route; Calculate the remaining mileage of the fleet based on the real-time position and the destination position; Calculate the travel time of the target fleet to the destination location based on the remaining mileage of the fleet, the average speed and the traffic operation status data; Based on the travel time, determining an estimated arrival time of the fleet; If the estimated arrival time of the convoy is earlier than the arrival time of the convoy, the predicted monitoring result generated is that the convoy arrives in advance; if the estimated arrival time of the convoy is later than the arrival time of the convoy, the predicted monitoring result generated is that the convoy arrives late.

7. The method according to claim 1, characterized in that The method further comprises: Determine whether there is a conflicting intersection on the security route, wherein the conflicting intersection is used to represent the intersection position of the security route of the target fleet and the security routes of other designated fleets; If yes, obtaining a second distance between the target convoy and the conflicting intersection, and a third distance between the remaining designated convoys and the conflicting intersection; Determining whether the second distance and the third distance are both less than a minimum conflict distance; If so, a conflict warning is generated and sent to the scheduling end.

8. The method according to claim 1, characterized in that: The method further comprises: Acquiring traffic data of the guaranteed route; Based on the traffic data, determining whether there is traffic congestion on the guaranteed route; If yes, obtain a GIS map of the area where the guaranteed route is located, and generate an alternative route based on the GIS map; The alternative routes are sent to the dispatching end to complete the optimization processing of the travel route of the target fleet.

9. A vehicle fleet traffic monitoring device based on multimodal data fusion, characterized in that: include: An acquisition unit, configured to acquire the guarantee route information of the target vehicle fleet, wherein the guarantee route information includes a guarantee route coordinate sequence, each route coordinate in the guarantee route coordinate sequence is respectively used to represent a turning point position in the guarantee route, and each route coordinate is sorted in order from the starting point to the end point of the guarantee route; A monitoring resource construction unit, configured to generate a monitoring resource sequence corresponding to the protection route according to the protection route coordinate sequence, wherein the monitoring resource sequence includes a plurality of road cameras, the monitoring range of the plurality of road cameras covers all inflection points in the protection route, and each inflection point corresponds to a road camera; A monitoring video generation unit, used to obtain the location information of the target vehicle fleet, and generate a real-time monitoring video of the target vehicle fleet based on the location information and the monitoring images of each road camera in the monitoring resource sequence; The sending unit is used to send the real-time monitoring video to the dispatching end, so as to complete the traffic monitoring of the target fleet after sending.

10. A vehicle fleet traffic monitoring device based on multimodal data fusion, characterized in that: include: A memory, a processor and a transceiver which are sequentially communicatively connected, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the fleet traffic monitoring method based on multimodal data fusion as described in any one of claims 1 to 8.

Images