Vehicle real-time monitoring method, device, equipment, storage medium and program product

Abstract

The invention provides a vehicle real-time monitoring method, device and equipment, a storage medium and a program product, and relates to the field of financial science and technology. The method comprises the following steps: responding to a live broadcast starting request of security personnel, and generating a push address; acquiring a real-time video of the vehicle from a server; inputting the real-time video into the target tracking model to generate a live video; and displaying the live video to a visual interface through the push address, thereby enabling security personnel to monitor the running condition of the vehicle through the live video. According to the scheme, the real-time video is processed through the target tracking model, the obtained live video is displayed to the visual interface through the push address, and meanwhile, the driving track, the driving speed and the driving time of the vehicle are displayed in the visual interface; therefore, security personnel can monitor the running condition of the vehicle in real time, and the running safety of the vehicle is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of financial technology, and in particular to a vehicle real-time monitoring method, device, equipment, storage medium and program product. BACKGROUND

[0002] With more and more customers choosing bank offline outlets to handle business, the demand for cash in outlets continues to increase, and it becomes increasingly important to ensure the safety and timeliness of transporting vehicles escorting cash to bank outlets.

[0003] The traditional vehicle escort method is to configure corresponding security personnel to follow the corresponding cash transport vehicle for escort, which consumes a large amount of human resources and the real-time running track and driving state of the cash transport vehicle cannot be mastered by bank security personnel. If the driving condition of the cash transport vehicle is abnormal, the bank security personnel cannot be immediately notified of the relevant situation.

[0004] Therefore, this method is difficult to monitor the running condition of the vehicle in real time. SUMMARY

[0005] The present application provides a vehicle real-time monitoring method, device, equipment, storage medium and program product to solve the technical problem of difficulty in monitoring the running condition of the vehicle in real time.

[0006] In a first aspect, the present application provides a vehicle real-time monitoring method, comprising: in response to a live broadcast start request of a security personnel, creating a live broadcast address and generating a push address according to the live broadcast address; obtaining real-time video of a vehicle from a server; inputting the real-time video into a target tracking model to generate a live broadcast video; and displaying the live broadcast video to a visual interface through the push address, so that the security personnel monitors the running condition of the vehicle through the live broadcast video.

[0007] In a possible implementation, inputting the real-time video into the target tracking model to generate the live broadcast video comprises: inputting the real-time video into a twin network to obtain video image features; generating a vehicle identification frame occurrence frequency proportion in a current video frame number and a monitoring video according to the video image features using a region proposal network; and generating the live broadcast video according to the vehicle identification frame occurrence frequency proportion in the current video frame number and the monitoring video.

[0008] In a possible implementation, it is judged whether the vehicle identification frame occurrence frequency proportion in the current video frame number is less than a preset threshold value; if the vehicle identification frame occurrence frequency proportion in the current video frame number is less than the preset threshold value, a warning information is generated and fed back to the security personnel.

[0009] In a possible implementation, in response to a live broadcast stop request of a security personnel; the live broadcast video and the security personnel account information are stored to a server.

[0010] In a possible implementation, the security personnel login information is received, it is judged whether the security personnel login information exists in the server, if the security personnel login information exists, the security personnel account information is acquired according to the security personnel login information, and the security personnel login information and the security personnel account information are verified to be consistent, then the login is successful, if the security personnel login information does not exist, the registration information is acquired and stored to the server.

[0011] In a possible implementation, before the real-time video of the vehicle is acquired from the server, the method further includes: determining the monitored vehicle, acquiring the real-time video of the vehicle shot by the unmanned aerial vehicle; and storing the real-time video of the vehicle to the server in time sequence.

[0012] In a possible implementation, after the real-time video of the vehicle is stored to the server in time sequence, the method further includes: screening real-time video frames according to a preset video frame rate ratio, and putting the real-time video frames into a buffer pool queue; wherein the preset video frame rate ratio represents a ratio between a real-time video frame rate and a preset video frame rate; judging whether the buffer pool queue reaches a preset storage capacity, if the buffer pool queue reaches the preset storage capacity, the earliest stored video frames in the buffer pool queue are cleared according to a preset clearance ratio.

[0013] In a possible implementation, the method further includes: creating a live broadcast address and generating a push address according to the live broadcast address in response to a live broadcast start request of the security personnel; acquiring the real-time video of the vehicle from the server; inputting the real-time video into a target tracking model to generate a live broadcast video; and displaying the live broadcast video to a visual interface through the push address, so that the security personnel monitors the running condition of the vehicle through the live broadcast video.

[0014] In a possible implementation, the acquiring module is further configured to: input the real-time video into a twin network to obtain video image features; generate a proportion of a number of times of appearance of a vehicle bounding box in a frame number of a current video and a monitoring video according to the video image features by using a region proposal network; and generate the live broadcast video according to the proportion of the number of times of appearance of the vehicle bounding box in the frame number of the current video and the monitoring video.

[0015] In a possible implementation, the processing module is further configured to: judge whether the proportion of the number of times of appearance of the vehicle bounding box in the frame number of the current video is less than a preset threshold; if the proportion of the number of times of appearance of the vehicle bounding box in the frame number of the current video is less than the preset threshold, generate an early warning information and feed back to the security personnel.

[0016] In a possible implementation, the processing module is further configured to: respond to a live broadcast stop request of the security personnel; and store the live broadcast video and the security personnel account information to the server.

[0017] In a possible implementation, the processing module is further configured to: receive security personnel login information, determine whether the security personnel login information exists in the server, acquire security personnel account information according to the security personnel login information if the security personnel login information exists, and verify that the security personnel login information and the security personnel account information are consistent, so that the login is successful, or acquire registration information and store the registration information in the server if the security personnel login information does not exist.

[0018] In a possible implementation, the acquiring module is further configured to: determine a monitored vehicle, acquire real-time video of the vehicle shot by the unmanned aerial vehicle, and store the real-time video of the vehicle in the server in chronological order.

[0019] In a possible implementation, the processing module is further configured to: filter real-time video frames according to a preset video frame rate ratio, and put the real-time video frames into a buffer pool queue, wherein the preset video frame rate ratio represents a ratio between a real-time video frame rate and a preset video frame rate, and determine whether the buffer pool queue reaches a preset storage capacity, and clear the earliest stored video frames in the buffer pool queue according to a preset clearance ratio if the buffer pool queue reaches the preset storage capacity.

[0020] In a third aspect, the present application provides an electronic device, comprising: a processor and a memory connected with the processor in communication; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to implement the method as above.

[0021] In a fourth aspect, the present application provides a computer readable storage medium, the computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by the processor to implement the method as above.

[0022] In a fifth aspect, the embodiments of the present application provide a computer program product, comprising a computer program, which is executed by the processor to implement the first aspect and / or various possible implementations of the first aspect as above.

[0023] The method, device, equipment, storage medium and program product provided by the present application for real-time monitoring of a vehicle, in response to a live broadcast starting request of a security personnel, create a live broadcast address and generate a push address according to the live broadcast address, acquire real-time video of a vehicle from a server, input the real-time video into a target tracking model to generate live broadcast video, and display the live broadcast video to a visual interface through the push address, so that the security personnel monitors the running condition of the vehicle through the live broadcast video. The present scheme processes real-time video through a target tracking model to obtain live broadcast video, which is displayed to a visual interface through a push address, and the running track, speed and time of the vehicle are displayed in the visual interface, so that the security personnel can monitor the running condition of the vehicle in real time, and the safety of vehicle running is improved. BRIEF DESCRIPTION OF DRAWINGS

[0024] The accompanying drawings, which are incorporated herein and constitute part of this specification, illustrate embodiments consistent with the application and serve to explain the principles

[0025] Figure 1 A flowchart of a vehicle real-time monitoring method provided by the application Figure 1 ;

[0026] Figure 2 A flowchart of a vehicle real-time monitoring system provided by the application Figure 1 ;

[0027] Figure 3 A flowchart of a vehicle real-time monitoring method provided by the application Figure 2 ;

[0028] Figure 4 A network structure diagram of a target tracking model provided by the application

[0029] Figure 5 A twin network part diagram of a target tracking model provided by the application

[0030] Figure 6 A region proposal network part diagram of a target tracking model provided by the application

[0031] Figure 7 A flowchart of a frame rate change solution provided by the application

[0032] Figure 8 A flowchart of a vehicle real-time monitoring system provided by the application Figure 2 ;

[0033] Figure 9 A structure diagram of a vehicle real-time monitoring device provided by the application Figure 1 ;

[0034] Figure 10 A structure diagram of an electronic device provided by the application

[0035] The specific embodiments of the application have been shown through the above drawings, and will be described in more detail hereinafter. These drawings and textual descriptions are not intended to limit the scope of the concept of the application by any means, but to illustrate the concept of the application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0036] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The following description is made with reference to the accompanying drawings in which like reference numerals represent like elements or similar elements, unless the context of the description dictates otherwise. The following description of exemplary embodiments is not representative of all embodiments consistent with the present application. Rather, it is merely an example of apparatus and methods consistent with some aspects of the present application as detailed in the appended claims.

[0037] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of related data comply with relevant laws, regulations and standards of relevant countries and regions, necessary security measures are taken, public order and good customs are not violated, and appropriate operation portals are provided for users to choose authorization or refusal.

[0038] And the present application involves big data analysis of user information (including but not limited to personal biological characteristics, identity data, consumption data, asset data, electronic terminal operation data, etc.), and uses artificial intelligence technology for automatic decision-making, and makes technical solutions based on automatic decision-making results that have a significant impact on personal rights and interests, provides appropriate operation portals for users to choose to agree or refuse automatic decision-making results; if the user chooses to refuse, the expert decision-making process is entered.

[0039] It should be noted that the method, device, equipment, storage medium and product for real-time monitoring of a vehicle provided by the present application can be used in the field of financial technology, and can also be used in any field other than financial technology. The application field of the method, device, equipment, storage medium and product for real-time monitoring of a vehicle in the present application is not limited.

[0040] Under the wave of digitization, although online financial services are booming, bank offline outlets still carry a large number of cash business demands as service entities; in recent years, with the continuous increase of customers' preference for offline handling of cash deposit, withdrawal and other businesses, the demand for cash reserves in outlets presents a stepwise growth trend, and it becomes more and more important to ensure the safety and timeliness of the transportation vehicles escorting cash to the bank outlets.

[0041] The traditional vehicle escort method is to follow the corresponding cash transport vehicle by configuring corresponding security personnel, that is, the mode of "security personnel following vehicle transportation + regular telephone report", and 3-5 security personnel need to be configured for each cash transport vehicle; a large amount of human resources is consumed; and the running track of the cash transport vehicle only relies on the oral report of the driver or the inefficient Global Positioning System (GPS), so that the bank security center cannot master the vehicle position, speed and driving state in real time. If the vehicle encounters traffic accidents, road blockades or tailing, the security personnel are difficult to detect in time, and the emergency disposal is delayed. In addition, due to the lack of real-time audio and video monitoring and vehicle state monitoring, the emergency plan cannot be started in the first time, which greatly increases the response time to risks, so that the bank is often in a passive situation when facing transportation risks, and it is difficult to ensure the safety of cash transportation.

[0042] Therefore, this method relying on traditional manual report and basic GPS positioning has information lag, risk of omission and difficulty in covering night and bad weather; the basic GPS positioning can only provide longitude and latitude data, and cannot capture the driving posture of the vehicle, the surrounding environment and the sudden abnormal situation; it is difficult to realize real-time monitoring of the running condition of the vehicle in all periods and high precision.

[0043] To solve the technical problem of difficult real-time monitoring of the running condition of the vehicle, the application provides a vehicle real-time monitoring method, device, equipment, storage medium and program product, which responds to a live broadcast starting request of a security personnel, creates a live broadcast address and generates a push address according to the live broadcast address, obtains real-time video of a vehicle from a server, inputs the real-time video into a target tracking model to generate live broadcast video, and displays the live broadcast video to a visual interface through the push address, so that the security personnel can monitor the running condition of the vehicle through the live broadcast video. The scheme processes the real-time video through the target tracking model to obtain the live broadcast video, which is displayed to the visual interface through the push address, and the driving track, driving speed and time of the vehicle are displayed in the visual interface; so as to monitor the running condition of the vehicle in real time, and improve the safety of vehicle operation.

[0044] The technical scheme of the application and how the technical scheme of the application solves the above technical problems will be described in detail in specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes can not be described again in some embodiments. The embodiments of the application will be described below with reference to the drawings.

[0045] Embodiment one

[0046] Figure 1 Flowchart of a vehicle real-time monitoring method provided by the application Figure 1 For example,Figure 2 As shown in the embodiments of this application, the method for real-time vehicle monitoring includes:

[0047] S101 responds to the security personnel's request to start the live stream, creates a live stream address, and generates a push address based on the live stream address;

[0048] For example, when security personnel initiate a "start live stream" request, the server responds by creating a live stream address (http: / / 123456) and a push address (rtmp: / / 123456) based on the request. When a security personnel initiates a live stream start request, the server responds by creating a live stream address based on the request and storing it in the live stream list on the server. The server maintains the live stream list to ensure that every address in it is a live stream; and a push address is generated based on the live stream address.

[0049] S102: Obtain real-time video of the vehicle from the server; input the real-time video into the target tracking model to generate live video;

[0050] The target tracking model is based on a Siamese network; the task of target tracking is to track the position, movement, and state of a specific target in a continuous sequence of images or videos. By using the target tracking model to process real-time vehicle video, precise vehicle positioning can be achieved, improving the accuracy of vehicle monitoring.

[0051] S103 displays live video to a visual interface via a push address, enabling security personnel to monitor vehicle operations through live video.

[0052] The system will generate a live video based on the proportion of the number of times the vehicle identification frame appears in the current video frame and the monitoring video, and display it on the visualization interface. At the same time, the visualization interface will display the vehicle's driving trajectory, speed and time.

[0053] In practical applications, Figure 1 A flowchart illustrating a real-time vehicle monitoring system provided in this application. Figure 2 ,like Figure 3 As shown, the vehicle real-time monitoring system includes a drone monitoring module, a video transmission module, a target tracking module, and a back-end management module. The drone monitoring module consists of drones that work in conjunction with the transport vehicles, primarily used to follow and capture real-time video of the vehicles' movement.

[0054] The video transmission module is primarily responsible for acquiring real-time video from the drone monitoring module and pushing the video to the target tracking module for monitoring and processing. Afterward, the video transmission module pulls the processed monitoring video from the monitoring module and pushes it to the backend management module for management and display. The specific real-time monitoring video transmission process mainly involves two steps: pushing and pulling. Pushing the stream refers to transmitting the monitoring video to the server, while pulling the stream refers to retrieving these video streams from the server for playback.

[0055] In practical applications, choosing a containerized platform to deploy server clusters can avoid the problem of the entire system becoming unusable due to the failure of a single server. Figure 2 A flowchart illustrating a method for real-time vehicle monitoring provided in this application. Figure 3 ,like Figure 4 As shown, the real-time video captured by the drone monitoring module is pushed to the server, video data is pulled from the server and sent to the target tracking module, and then the video is provided to the target tracking model for processing. The resulting video is pushed back to the server and finally displayed and managed in the monitoring management interface.

[0056] This application provides a method, apparatus, device, storage medium, and program product for real-time vehicle monitoring. Responding to a live-streaming request from security personnel, the system creates a live-streaming address and generates a push address based on the live-streaming address; it retrieves real-time vehicle video from a server; inputs the real-time video into a target tracking model to generate a live-streaming video; and displays the live-streaming video on a visualization interface via the push address, allowing security personnel to view the live-streaming video. This solution processes real-time video using a target tracking model, and the resulting live-streaming video is displayed on a visualization interface via the push address. The visualization interface also displays the vehicle's trajectory, speed, and time, enabling security personnel to monitor vehicle operation in real time and improving vehicle safety.

[0057] In one possible implementation, real-time video is input into the target tracking model to generate live video, including:

[0058] Real-time video is input into a twin network to obtain video image features; based on the video image features, a region proposal network is used to generate the proportion of vehicle identification box occurrences in the current video frame and the monitoring video.

[0059] A live video is generated based on the proportion of vehicle identification frame occurrences in the current video frame count and the monitoring video.

[0060] Wherein, the vehicle identification frame is used to mark the position of the vehicle in the video picture; the vehicle identification frame occurrence number is, for example, in each frame of the video, if the vehicle is detected, that is, there is a vehicle identification frame, it is counted as 1 time of 'occurrence'; if there is no vehicle identification frame in a frame of the video, it is counted as 'non-occurrence'; the proportion of the vehicle identification frame occurrence number in the frame number of the current video is calculated by the following formula: For example, the real-time video input into the target tracking model is 1000 frames, and the vehicle identification frame is detected in 750 frames, so the proportion of the vehicle identification frame occurrence number in the frame number of the current video is: .

[0061] The Siamese network is a neural network architecture for solving the similarity measurement problem. Figure 4 The network structure diagram of the target tracking model provided by the application is shown in Fig. 1. Figure 5 As shown in Fig. 1, Siamese Network represents the Siamese network, Region Proposal Network represents the region proposal network, Template Frame represents the template frame, Detection Frame represents the detection frame, Classification Branch represents the classification branch for distinguishing the target and the background, Regression Branch represents the regression branch for fine-tuning the candidate region, Positive represents the positive sample, Negative represents the negative sample, One Group represents the single group, KGroups represents the multiple groups, and dx, dy, dw and dh all represent the offset vector. The real-time video is input into the target tracking model for processing; specifically, the real-time video is input into the Siamese network to obtain the video image feature; the proportion of the vehicle identification frame occurrence number in the frame number of the current video and the monitoring video are generated by using the region proposal network according to the video image feature.

[0062] In addition, in order to improve the accuracy of target tracking and positioning, the target tracking model combines multi-layer feature fusion. Figure 5 The Siamese network part of the target tracking model provided by the application is shown in Fig. 2. Figure 6 As shown in Fig. 2, the Siamese network part of the target tracking model is constructed based on a deep learning model; crop represents cropping, that is, extracting a specific region from an image, such as random cropping and center cropping; conv represents convolution, which is used to extract image features; and concat represents concatenation, that is, concatenating multiple image features.

[0063] Figure 6 The region proposal network part of the target tracking model provided by the application is shown in Fig. 3. Figure 7As shown, two feature maps are input on the left side: small scale: 6x6x512, which contains high-level semantics, and large scale: 22x22x512, which contains low-level details; multi-scale features are extracted using different size convolution kernels, specifically, the small scale feature map generates 4x4x(2kx512), 20x20x512, etc. feature maps, and the large scale feature map generates 4x4x(4kx512), 20x20x512, etc. feature maps; the obtained multi-scale feature maps are input into the star module in the figure for feature fusion; 17x17x2k feature maps and 17x17x4k feature maps are obtained; the branch classification and regression branch are used for processing respectively.

[0064] In actual application, the real-time video input twin network extracts multi-level features through a multi-level feature extraction module, performs pixel-level analysis and abstraction on each frame of picture, accurately captures detailed information such as vehicle contour, license plate texture, and vehicle body painting, and generates high-dimensional and multi-level video image feature vectors. The feature vectors output by the twin network are input into the region proposal network, which uses an anchor box mechanism and a lightweight convolution kernel to scan the feature map with a sliding window to determine the vehicle identification box; after optimization by the non-maximum suppression algorithm, the frequency of the vehicle identification box in the full frame of the video is counted in real time, the proportion of the number of vehicle identification box occurrences in the frame number of the current video is accurately calculated, and the complete monitoring video is output,

[0065] The twin network extracts image features to ensure that the obtained features have strong expression; the region proposal network generates identification boxes according to the features to improve the generation accuracy of the standard box; by combining the twin network and the region proposal network to process real-time video, the accuracy of the proportion of the number of vehicle identification box occurrences in the frame number of the current video is improved.

[0066] In one possible implementation, it is determined whether the proportion of the number of vehicle identification box occurrences in the frame number of the current video is less than a preset threshold value.

[0067] If the proportion of the number of vehicle identification box occurrences in the frame number of the current video is less than the preset threshold value, a warning information is generated and fed back to the security personnel.

[0068] The preset threshold is set according to requirements, for example, the preset threshold is set as 85%; for example, in the current transportation task, the proportion of the number of vehicle identification frame appearances in the frame number of the current video is 75%; by comparison, if the proportion of the number of vehicle identification frame appearances in the frame number of the current video is less than the preset threshold, the early warning mechanism is triggered; a visual early warning report is generated, and the abnormal period and specific frame number are marked; warning information is sent to the security personnel through multi-terminal pushing such as security management application pop-up window, short message, vehicle-mounted central control screen, etc., prompting the security personnel that the driving condition of the vehicle is abnormal; whether the vehicle operation is abnormal is determined by detecting the proportion of the number of vehicle identification frame appearances in the frame number of the current video, which can alert the security personnel and make timely response, and ensure the safety of vehicle operation.

[0069] In a possible implementation, in response to the live broadcast stop request of the security personnel; the live broadcast video and the security personnel account information are stored to the server.

[0070] When the security personnel stops watching the live broadcast, in response to the live broadcast stop request of the security personnel; the live broadcast video is named according to the timestamp and the event number to generate a video file; at the same time, the account information of the security personnel is called, including the operation permission, the belonging security group, the login time and other key data, which form a corresponding relationship with the live broadcast video; the video file and the account information are stored in the live broadcast list of the server, and according to the above example, the live broadcast video and the security personnel account information are stored; so as to facilitate the security personnel to query and view the video file later, so as to analyze the vehicle running condition and improve the driving route.

[0071] In a possible implementation, the security personnel login information is received, and it is judged whether the security personnel login information exists in the server;

[0072] If the security personnel login information exists, the security personnel account information is obtained according to the security personnel login information; and the security personnel login information and the security personnel account information are verified to be consistent, and then the login is successful;

[0073] If the security personnel login information does not exist, the registration information is obtained and stored in the server.

[0074] In combination with the previous example, only the security personnel who successfully log in have the right to request to watch the live broadcast. For example, security personnel A, employee ID: BH00123; opens the security management application, enters the account A_bh and password, and clicks to log in. The security management application sends the login information of security personnel A: account, password to the background server; if the account A_bh of security personnel A exists in the server, the account information of security personnel A is obtained; the password in the account information and the login information is verified to be consistent; if consistent, the login is successful. If the account A_bh of security personnel A does not exist in the server, the registration information of security personnel A is collected to generate account information and stored in the server, wherein the registration information includes: account, password, real name, employee ID, contact information, etc.

[0075] Through the verification of the login information, it is ensured that the login person holds a legal identity certificate. Avoiding non-security personnel using a virtual account to log in, as there is no corresponding record in the server, it can be directly intercepted, resulting in unauthorized personnel obtaining confidential information such as escort routes.

[0076] In one possible implementation, before obtaining the real-time video of the vehicle from the server, further comprising:

[0077] Determine the monitored vehicle, obtain the real-time video of the vehicle shot by the unmanned aerial vehicle; store the real-time video of the vehicle in time sequence to the server.

[0078] Specifically, the cash transport vehicle is determined as the monitored vehicle 1; the unmanned aerial vehicle is called to lock the driving track of the vehicle 1 in real time through the Beidou positioning system and the artificial intelligence visual recognition technology, and the real-time video is shot; the real-time video obtained by shooting is stored in time sequence to the server. In actual application, the unmanned aerial vehicle is configured to carry a 4K or 8K ultra-high-definition camera, supports optical image stabilization and automatic zooming, and can ensure that the picture of the high-speed driving vehicle is clear; in the face of night or low light scene, the unmanned aerial vehicle automatically switches to infrared thermal imaging mode, is equipped with high-precision thermal sensor, can accurately identify vehicle engine, tire and other components, and supports dual-mode picture fusion, superimposes infrared thermal map and visible light image, retains vehicle appearance features, and highlights heat source distribution, providing multi-dimensional data support for monitoring and analysis.

[0079] The way of using the unmanned aerial vehicle to shoot the transport vehicle in real time realizes full-view coverage; at the same time, based on the anti-interference performance of the unmanned aerial vehicle, when electromagnetic interference is encountered, the frequency band is automatically switched and the link redundancy technology is enabled, to ensure that the video transmission is not interrupted.

[0080] In one possible implementation, after storing the real-time video of the vehicle in time sequence to the server, further comprising:

[0081] The real-time video frames are filtered according to a preset video frame rate ratio, and the real-time video frames are put into a buffer pool queue; wherein the preset video frame rate ratio represents the ratio between the real-time video frame rate and the preset video frame rate;

[0082] It is judged whether the buffer pool queue reaches a preset storage capacity, and if so, the earliest stored video frames in the buffer pool queue are cleared according to a preset clearance ratio.

[0083] In actual target tracking algorithms, there is often a difference between the frame rate of the video and the running rate of the algorithm. Over time, this leads to an increase in the delay between the monitoring video processed by the target tracking module and the original video recorded by the unmanned aerial vehicle. To solve this problem, a buffer pool queue is set to accommodate incompatible frame rates; Figure 7 A flowchart of a solution method for changing the frame rate provided by the present application is shown in FIG. Figure 8 As shown, real-time video in a server is obtained, real-time video frames are filtered from the real-time video according to a preset video frame rate ratio, and the filtered real-time video frames are put into the end of the buffer pool queue; when the queue is full, the video frames at the head of the buffer pool queue are cleared according to a preset clearance ratio. For example, the preset video frame rate is set to 10 frames per second, and the real-time video frame rate is 20 frames per second; the preset video frame rate ratio is calculated according to the real-time frame rate and the preset frame rate, i.e. 20 ÷ 10 = 2, which means that only 1 frame is retained in every 2 real-time video frames. The preset storage capacity is 1000 frames of video frames; the preset clearance ratio is set to 20%, i.e. when the buffer pool queue storage reaches 1000 frames, 20% of the earliest stored video frames are cleared. When the real-time video of the vehicle is obtained from the server, the video frames are obtained from the buffer pool queue to form the real-time video.

[0084] Specifically, to ensure that the live video remains extremely smooth during transmission and playback, audio and video processing tools can be used to optimize the live source in real time: at the encoding end, automatically adapt to high compression ratio encoding protocols to significantly reduce the video code rate while ensuring image quality; through dynamic frame rate adjustment technology, the video frame rate is adjusted in real time according to the network bandwidth to avoid lag; combined with intelligent key frame optimization algorithm, reduce video transmission redundant data. The live stream processed by the audio processing tool is transmitted at high speed to the visualization interface through the generated push address.

[0085] Figure 2 A flowchart of a vehicle real-time monitoring system provided by the present application is shown in FIG. Figure 8 ; as Figure 9The real-time video of the vehicle photographed by the unmanned monitoring module is pushed to the server of the video transmission module for storage. After the real-time video is read from the server and sent to the target tracking model to generate a live video, the live video is sent to the video processing tool to improve the smoothness of the live video, and then the live video is pushed to the server. In response to a live start request of the security personnel, the live video is obtained from the server and sent to the video stream distributor to obtain the vehicle running track and the driving time and speed. At the same time, the proportion of the number of times of the generated vehicle identification frame in the frame number of the current video is obtained from the target tracking module and uploaded to the recorder for recording. The live video, the vehicle running track and the driving time and speed, and the proportion of the number of times of the vehicle identification frame in the frame number of the current video are displayed to the visualization interface.

[0086] Embodiment two

[0087] Figure 1 A structure diagram of a vehicle real-time monitoring device provided by the present application Figure 9 As shown in the figure, Figure 10 The device 900 for real-time monitoring of vehicles provided by the embodiments of the present application comprises:

[0088] The creation module 901 is configured to create a live address and generate a push address according to the live address in response to a live start request of the security personnel.

[0089] The acquisition module 902 is configured to acquire the real-time video of the vehicle from the server, input the real-time video into the target tracking model, and generate a live video.

[0090] The processing module 903 is configured to display the live video to the visualization interface through the push address, so that the security personnel can monitor the running condition of the vehicle through the live video.

[0091] The device for real-time monitoring of vehicles provided by the embodiments of the present application can execute the method provided by the above-mentioned method embodiments, and has similar implementation principles and technical effects. Here, the device for real-time monitoring of vehicles will not be described in detail.

[0092] Figure 10 A structure diagram of an electronic device provided by the present application. As shown in the figure, ​ The electronic device 100 provided by the embodiments of the present application comprises at least one processor 1001 and a memory 1002. Optionally, the device 100 further comprises a communication component 1003. The processor 1001, the memory 1002 and the communication component 1003 are connected through a bus 1004.

[0093] In the specific implementation process, the at least one processor 1001 executes the computer execution instructions stored in the memory 1002, so that the at least one processor 1001 executes the above-mentioned method.

[0094] The specific implementation process of the processor 1001 can refer to the method embodiments described above, which have similar implementation principles and technical effects, and will not be described here.

[0095] The present application also provides a computer-readable storage medium, which stores computer execution instructions, and when the processor executes the computer execution instructions, the method described above is implemented; and the implementation principle and technical effects are similar, and will not be described here.

[0096] The present application also provides a computer program product, which includes a computer program, and when the processor executes the computer program, the technical solution of the method embodiment described above is implemented, and the implementation principle and technical effects are similar, and will not be described here.

[0097] It should be noted that, for the above-mentioned method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the action order described, because according to the present application, some steps can be performed in other order or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0098] It should be further noted that, although each step in the flowchart is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise stated in this paper, the execution of these steps has no strict order limit, and these steps can be executed in other order. Moreover, at least part of the steps in the flowchart can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily sequential, but can be executed with at least part of other steps or other steps or stages of sub-steps or stages.

[0099] It should be understood that the device embodiments described above are only schematic, and the device of the present application can also be realized by other ways. For example, the division of units / modules in the above embodiments is only a logical function division, and actual implementation can have another division way. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.

[0100] In addition, each functional unit / module in the embodiments of the present application can be integrated in one unit / module, or each unit / module can exist physically, or two or more units / modules can be integrated together. The integrated unit / module can be realized in the form of hardware or in the form of a software program module.

[0101] The integrated unit / module, if realized in the form of hardware, can be a digital circuit, an analog circuit, etc. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. Unless otherwise specified, the processor can be any appropriate hardware processor, such as a CPU, a GPU, an FPGA, a DSP, an ASIC, etc. Unless otherwise specified, the storage unit can be any appropriate magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory (RRAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), an enhanced dynamic random access memory (EDRAM), a high-bandwidth memory (HBM), a hybrid memory cube (HMC), etc.

[0102] The integrated unit / module, if realized in the form of a software program module and sold or used as an independent product, can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application, essentially or the part that contributes to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the embodiments of the method of the present application. The aforementioned storage medium includes: a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various other media that can store program codes.

[0103] In the above embodiments, the description of each of the embodiments focuses on different aspects of the embodiments. The parts not described in detail in a certain embodiment can be seen in the relevant description of the other embodiments. The technical features of the above embodiments can be combined in any manner. In order to make the description simple, not all possible combinations of the technical features in the above embodiments are described, however, as long as the combinations of the technical features do not exist contradictory, they should be considered as falling within the scope of the disclosure.

[0104] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the application being indicated by the following claims.

[0105] It should be understood that the application is not limited to the precise construction and combinations of the components and steps described above and shown in the accompanying drawings. The scope of the application is limited only by the claims that follow.

Claims

1. A method for real-time monitoring of a vehicle, characterized in that, The method comprises: in response to a live broadcast starting request of a security personnel, creating a live broadcast address and generating a push address according to the live broadcast address; obtaining real-time video of a vehicle from a server; inputting the real-time video into a target tracking model to generate a live broadcast video; displaying the live broadcast video to a visual interface through the push address, so that the security personnel monitors the running condition of the vehicle through the live broadcast video.

2. The method of claim 1, wherein, The method further comprises: determining whether the proportion of the number of times of appearance of the vehicle identification frame in the frame number of the current video is less than a preset threshold value; if the proportion of the number of times of appearance of the vehicle identification frame in the frame number of the current video is less than the preset threshold value, generating an early warning information and feeding back to the security personnel.

3. The method of claim 2, wherein, The method further comprises: in response to a live broadcast stopping request of a security personnel; storing the live broadcast video and the security personnel account information to the server. The method further comprises:

4. The method of claim 1, wherein, receiving security personnel login information, and determining whether the security personnel login information exists in the server; if the security personnel login information exists, obtaining security personnel account information according to the security personnel login information; and verifying that the security personnel login information and the security personnel account information are consistent, then the login is successful; 5. The method of claim 4, wherein, if the security personnel login information does not exist, obtaining registration information and storing the registration information to the server. Before the real-time video of the vehicle is obtained from the server, the method further comprises: determining a monitored vehicle, obtaining real-time video of the vehicle shot by a drone; and storing the real-time video of the vehicle to the server in chronological order. After the real-time video of the vehicle is stored to the server in chronological order, the method further comprises:

6. The method of claim 1, wherein, screening real-time video frames according to a preset video frame rate ratio, and putting the real-time video frames into a buffer pool queue; wherein the preset video frame rate ratio represents the ratio between the real-time video frame rate and the preset video frame rate; determining whether the buffer pool queue reaches a preset storage capacity, if the buffer pool queue reaches the preset storage capacity, clearing the earliest stored video frames in the buffer pool queue according to a preset clearance ratio.

7. The method of claim 6, wherein, The method comprises: a creating module, configured to create a live broadcast address and generate a push address according to the live broadcast address in response to a live broadcast starting request of a security personnel; an obtaining module, configured to obtain real-time video of a vehicle from a server; 8. A device for real-time monitoring of a vehicle, characterized in that inputting the real-time video into a target tracking model to generate a live broadcast video; a processing module, configured to display the live broadcast video to a visual interface through the push address, so that the security personnel monitors the running condition of the vehicle through the live broadcast video. The method comprises: a memory and a processor; the memory stores computer execution instructions; 9. An electronic device, comprising: the processor executes the computer execution instructions stored in the memory, so that the processor executes the method according to any one of claims 1-7. The method comprises: a memory and a processor; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory, so that the processor executes the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer-executable instructions which, when executed by the processor, implement the method of any one of claims 1-7.

11. A computer program product, characterised in that, A computer program which, when executed by the processor, implements the method of any one of claims 1-7.

Images