Vehicle speed detection and collision warning method and electronic device

Abstract

The present disclosure relates to the field of computer vision, and particularly relates to a vehicle speed detection and collision warning method, an electronic device and a vehicle warning system; a vehicle speed detection method is provided, comprising: acquiring a monitoring video of a road, and extracting a plurality of continuous images in the monitoring video; identifying vehicles in the plurality of images, and establishing a driving track of a target vehicle; acquiring image position coordinates of the target vehicle in each image; obtaining world position coordinates of the target vehicle in a real world according to the image position coordinates; calculating a moving distance of the target vehicle in the real world between each two adjacent images in the plurality of images according to the world position coordinates; and calculating a speed of the target vehicle in a current frame according to a driving distance of the target vehicle in the real world and a frame rate of the monitoring video, wherein the driving distance is obtained according to the moving distance.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of computer vision, and in particular, to a vehicle speed detection and collision warning method, an electronic device and a vehicle warning system. BACKGROUND

[0002] Target vehicles driving on the road are detected by interval speed measurement and radar speed measurement. Interval speed measurement is to set two adjacent monitoring points on the same road section, calculate the average driving speed of the target vehicle on the road section based on the time of the target vehicle passing through the two monitoring points, and determine whether the speed is exceeded based on the comparison between the average driving speed and the reference speed.

[0003] Radar speed measurement mainly uses the principle of Doppler Effect: when the target approaches the radar antenna, the reflected signal frequency will be higher than the transmitter frequency; on the contrary, when the target moves away from the antenna, the reflected signal frequency will be lower than the transmitter frequency. Radar speed measurement can detect the instantaneous speed of the target vehicle. SUMMARY

[0004] In one aspect, the present disclosure provides a vehicle speed detection method, comprising: acquiring a monitoring video of a road, and extracting a plurality of continuous frames of images in the monitoring video. Identifying vehicles in the plurality of frames of images, and establishing a driving track of a target vehicle. Obtaining image position coordinates of the target vehicle in each frame of image. According to the image position coordinates, obtaining world position coordinates of the target vehicle in the real world. According to the world position coordinates, calculating a moving distance of the target vehicle in the real world in each adjacent two frames of images in the plurality of frames of images. According to the driving distance of the target vehicle in the real world and the frame rate of the monitoring video, calculating the speed of the target vehicle in the current frame, wherein the driving distance is obtained according to the moving distance.

[0005] In some embodiments, according to the driving distance of the target vehicle in the real world and the frame rate of the monitoring video, the speed of the target vehicle in the current frame is calculated, comprising: calculating the sum of the moving distances of the target vehicle in the real world in each adjacent two frames of images in the current frame of image and the N frames of images before the current frame of image, and the sum of the moving distances as the driving distance. According to the frame rate of the monitoring video, obtaining the time difference between the current frame of image and the Nth frame of image before the current frame of image. According to the driving distance and the time difference, obtaining the calculated speed of the target vehicle in the current frame, and taking the calculated speed of the target vehicle in the current frame as the speed of the target vehicle in the current frame.

[0006] In some embodiments, the method further comprises: calculating a distance of movement of the target vehicle in the real world in each pair of adjacent frames of the plurality of frames of images according to the world position coordinates; and determining whether the distance of movement of the target vehicle in the real world in each pair of adjacent frames of the plurality of frames of images is greater than a distance threshold value. Based on the distance of movement of the target vehicle in the real world in each pair of adjacent frames of the plurality of frames of images being greater than the distance threshold value, the speed of the target vehicle in the current frame is calculated; and based on the distance of movement of the target vehicle in the real world in each pair of adjacent frames of the plurality of frames of images being less than or equal to the distance threshold value, the driving trajectory of the target vehicle is tracked.

[0007] In some embodiments, before the distance of movement of the target vehicle in the real world in each pair of adjacent frames of the plurality of frames of images is calculated according to the world position coordinates, the method further comprises: calculating a distance of movement of the target vehicle in each frame of the plurality of frames of images; and determining whether the distance of movement of the target vehicle in each frame of the plurality of frames of images is greater than a distance threshold value. Based on the distance of movement of the target vehicle in each frame of the plurality of frames of images being greater than the distance threshold value, the distance of movement of the target vehicle in the real world in each pair of adjacent frames of the plurality of frames of images is calculated; and based on the distance of movement of the target vehicle in each frame of the plurality of frames of images being less than or equal to the distance threshold value, the driving trajectory of the target vehicle is tracked.

[0008] In some embodiments, before the speed of the target vehicle in the current frame is calculated, the method further comprises: calculating a number of image movement distances greater than a distance threshold value in the plurality of frames of images before the current frame of image, wherein each image movement distance is a distance of movement of the target vehicle in each pair of adjacent frames of images; and determining whether the number of image movement distances greater than the distance threshold value is greater than a set threshold value. Based on the number of image movement distances greater than the distance threshold value being greater than the set threshold value, the speed of the target vehicle in the current frame is calculated; and based on the number of image movement distances greater than the distance threshold value being less than or equal to the set threshold value, the driving trajectory of the target vehicle is tracked.

[0009] In some embodiments, the vehicle speed detection method further comprises: determining whether the speed of the target vehicle in each of the L consecutive frames of images is outside a driving speed limit range of a road on which the target vehicle is driving. Based on the speed of the target vehicle in each of the L consecutive frames of images being outside the driving speed limit range of the road on which the target vehicle is driving, a pre-warning operation is performed; and based on the speed of the target vehicle in at least one of the L consecutive frames of images being within the driving speed limit range of the road on which the target vehicle is driving, the pre-warning operation is not performed. When the target vehicle triggers a pre-warning operation during speed detection in each of the plurality of monitoring videos, the warning intensity is increased when the target vehicle triggers a pre-warning operation during subsequent speed detection.

[0010] In some embodiments, the vehicle speed detection method further comprises, before identifying the target vehicle in the plurality of images and establishing the driving track of the target vehicle, marking a detection region on each of the plurality of images. The detection region is a closed figure located in a driving area of the road in the image, and the boundary of the detection region has fixed image position coordinates in each of the plurality of images. The moving distance of the target vehicle in the real world between two adjacent images in the plurality of images is calculated according to the world position coordinates, including: calculating the moving distance of the target vehicle located in the detection region in the real world between two adjacent images in the plurality of images according to the world position coordinates. The speed of the target vehicle in the current frame is calculated, including: calculating the speed of the target vehicle located in the detection region in the current frame.

[0011] In some embodiments, identifying the target vehicle in the plurality of images and establishing the driving track of the target vehicle comprises: detecting the plurality of images to determine the target vehicle and establish a target bounding box, tracking the target vehicle based on the target bounding box to obtain the motion track of the target vehicle. The image position coordinates of the target vehicle in each of the plurality of images are the image position coordinates of the center point of the target bounding box of the target vehicle in the frame.

[0012] In some embodiments, the vehicle speed detection method further comprises: after establishing the driving track of the target vehicle, establishing an identity information list of the target vehicle. Establishing the identity information list of the target vehicle comprises: extracting a feature vector of the target vehicle using a re-identification model. Calculating the cosine of the angle of the feature vector of the target vehicle between each two adjacent images. Determine whether the value of the cosine of the angle is greater than a similarity threshold value for G consecutive times. Based on the value of the cosine of the angle being greater than the similarity threshold value for G consecutive times, establish an identity information list of the corresponding target vehicle in the vehicle information retrieval library, and store the feature vector of the target vehicle in the identity information list of the corresponding target vehicle. The identity information list of the target vehicle includes the identity information of the target vehicle.

[0013] In some embodiments, in the case that the target vehicle tracking is lost, it is determined whether the target bounding box of the target vehicle is located in the detection region in the frame image before the target vehicle tracking is lost. Based on the target bounding box of the target vehicle being located in the detection region, the feature vector of the lost target vehicle is matched with the feature vector of the newly acquired target vehicle after the tracking is lost, and the identity information of the newly acquired target vehicle that matches the feature vector is established in the identity information list of the lost target vehicle. Based on the target bounding box of the target vehicle being located outside the detection region, the search for the target vehicle is stopped.

[0014] In some embodiments, the multi-frame images are detected, the target vehicle is determined and the target detection frame is established, the target vehicle is tracked based on the target detection frame, and the motion trajectory of the target vehicle is obtained. The method further comprises: there are multiple target detection frames, and in the case that two or more target detection frames overlap, the feature vectors of the target vehicles corresponding to the overlapping target detection frames are extracted. The target vehicles with a cosine value of the feature vectors greater than a similarity threshold in adjacent two frames of images are established in the same identity information list.

[0015] In some embodiments, before obtaining the world position coordinates of the target vehicle in the real world according to the image position coordinates, the method further comprises: calculating the internal parameters and the external parameters of the image acquisition device used for shooting the monitoring video. The internal parameters and the external parameters are used for converting the image position coordinates of the multi-frame images and the corresponding world position coordinates. The calculation of the internal parameters and the external parameters of the image acquisition device used for shooting the monitoring video comprises: labeling a first vanishing point and a second vanishing point on a labeled image in the multi-frame images. The labeled image is any frame in the multi-frame images. The image position coordinates of the first vanishing point and the second vanishing point in the labeled image are obtained. A straight line equation passing through the first vanishing point and the second vanishing point is established. The center of the labeled image is coincided with the principal point, and the initial internal parameters and the initial external parameters of the image acquisition device are calculated according to the straight line equation. At least one calibration reference is selected on the labeled image. The calibration reference is a marker with a known distance between the two ends in the real world. The calibration reference comprises a line segment of a dashed lane line, an interval line between adjacent dashed lane lines, and an interval line between two connected line segments in the same dashed lane line. The image position coordinates of the two end points of the calibration reference in the labeled image are obtained. At least one calibration reference is used as a constraint condition to construct a constraint formula, and the initial internal parameters and the initial external parameters are iterated according to the constraint formula. The internal parameters and the external parameters of the image acquisition device are obtained according to the optimal solution of the constraint formula.

[0016] The constraint formula is:

[0017]

[0018] wherein N is the number of calibration references, P K is the world position coordinates of one end of the kth calibration reference in the real world, Q K is the world position coordinates of the other end of the kth calibration reference in the real world; is the image position coordinates of one end of the kth calibration reference in the labeled image, the world position coordinates of the real world are calculated by using the initial internal parameters and the initial external parameters, Another end of the k-th calibration reference in the annotated image is an image position coordinate, and a real-world world position coordinate calculated by using the initial intrinsic parameters and the initial extrinsic parameters; cp represents constraint parameters of the image acquisition device, including the intrinsic parameters and the extrinsic parameters.

[0019] In another aspect, the present disclosure provides a vehicle collision warning method, wherein the vehicle collision warning method comprises using the vehicle speed detection method in any one of the above aspects, acquiring a monitoring video of a road and extracting a plurality of continuous frames of images in the monitoring video. The target vehicle is subjected to target detection and tracking, and speed detection. The vehicle collision warning method further comprises: establishing motion trajectories of a plurality of target vehicles in the monitoring video of the road. It is determined whether the motion trajectories of at least two adjacent target vehicles in the plurality of target vehicles are the same motion trajectory. If the motion trajectories of the two adjacent target vehicles are the same motion trajectory, the speeds of the two adjacent target vehicles in each frame of the plurality of frames of images are detected, and it is determined whether the speed of the rear target vehicle is continuously greater than the speed of the front target vehicle within a preset duration. Based on the speed of the rear target vehicle being continuously greater than the speed of the front target vehicle within the preset duration, a vehicle collision warning operation is performed. Based on the speed of the rear target vehicle being less than or equal to the speed of the front target vehicle within the preset duration, the vehicle collision warning operation is not performed.

[0020] In some embodiments, the vehicle collision warning method further comprises: if the motion trajectories of the W adjacent target vehicles are the same motion trajectory, the vehicle types of the W adjacent target vehicles are acquired, and W is greater than or equal to 3. It is determined whether the vehicle types of the W adjacent target vehicles are at least one small or medium-sized vehicle located between two large-sized vehicles. Based on the presence of at least one small or medium-sized vehicle located between the two large-sized vehicles, the vehicle collision warning operation is performed; based on the absence of at least one small or medium-sized vehicle located between the two large-sized vehicles, the speeds of the W adjacent target vehicles in each frame of the plurality of frames of images are detected, and it is determined whether the speed of the rear target vehicle is continuously greater than the speed of the front target vehicle within a preset duration. Based on the speed of the rear target vehicle being continuously greater than the speed of the front target vehicle within the preset duration, the vehicle collision warning operation is triggered; based on the speed of the rear target vehicle being less than or equal to the speed of the front target vehicle within the preset duration, the vehicle collision warning operation is not performed.

[0021] In some embodiments, the method for determining whether the motion trajectories of at least two target vehicles are the same motion trajectory comprises: obtaining a set of image position coordinates of each target vehicle in multiple frames of images, fitting a straight line equation to obtain a motion straight line equation of each target vehicle in an image coordinate system, wherein the origin of the image coordinate system coincides with the center of each frame of image. Determining whether the difference between the slopes of at least two motion straight line equations is less than a slope threshold value, and the difference between the intercepts of at least two motion straight line equations is less than an intercept threshold value. Based on the difference between the slopes of at least two motion straight line equations being less than the slope threshold value, and the difference between the intercepts of at least two motion straight line equations being less than the intercept threshold value, determining that the motion trajectories of at least two target vehicles corresponding to at least two motion straight line equations are the same motion trajectory.

[0022] In some embodiments, after the same target vehicle triggers a collision warning in a collision warning of multiple monitoring videos, when the target vehicle triggers a collision warning again in the collision warning, the alarm intensity is increased.

[0023] In another aspect, the present disclosure provides an electronic device comprising: a processor and a memory. The processor is configured to perform the following steps: obtaining a monitoring video of a road and storing the monitoring video to the memory; extracting multiple consecutive frames of images in the monitoring video. Identifying a target vehicle in the multiple frames of images and establishing a driving trajectory of the target vehicle. Obtaining image position coordinates of the target vehicle in each frame of image. Obtaining world position coordinates of the target vehicle in a real world according to the image position coordinates. Calculating a moving distance of the target vehicle in the real world between two adjacent frames of images of the monitoring video of the road. Calculating a speed of the target vehicle in a current frame according to a driving distance of the target vehicle in the real world and a frame rate of the monitoring video, wherein the driving distance is obtained according to the moving distance.

[0024] In some embodiments, the processor is further configured to perform the following steps: calculating a sum of moving distances of the target vehicle in the real world between each adjacent two frames of images in the current frame of image and N frames of images before the current frame of image, and the sum of moving distances is used as the driving distance. Obtaining a time difference between the current frame of image and the Nth frame of image before the current frame of image according to the frame rate of the monitoring video. Obtaining a calculated speed of the target vehicle in the current frame according to the driving distance and the time difference, and using the calculated speed of the target vehicle in the current frame as the speed of the target vehicle in the current frame.

[0025] In some embodiments, the processor is further configured to perform the following steps: calculate a sum of moving distances of the target vehicle in the real world in each pair of adjacent frames of the current frame image and N frame images before the current frame image, the sum of moving distances being a driving distance. According to a frame rate of the monitoring video, a time difference between the current frame image and the Nth frame image before the current frame image is obtained. According to the driving distance and the time difference, a calculated speed of the target vehicle in the current frame is obtained. The calculated speed of the target vehicle in the current frame is data smoothed with calculated speeds of the target vehicle in each of the M frame images before the current frame image, and a result of the data smoothing is obtained as a speed of the target vehicle in the current frame.

[0026] In some embodiments, before the processor is configured to calculate the moving distance of the target vehicle in the real world in each pair of adjacent frames of the multiple frame images according to the world position coordinates, the processor is further configured to perform the following steps: calculate a moving distance of the target vehicle in the image in each pair of adjacent frames of the monitoring video of the road. Determine whether the moving distance of the target vehicle in the image is greater than a distance threshold. Based on the moving distance of the target vehicle in the image being greater than the distance threshold, calculate the moving distance of the target vehicle in the real world; based on the moving distance of the target vehicle in the image being less than or equal to the distance threshold, continue to track the driving trajectory of the target vehicle.

[0027] In some embodiments, before the processor is configured to calculate the speed of the target vehicle in the current frame, the processor is further configured to perform the following steps: calculate a number of image moving distances greater than a distance threshold in a plurality of image moving distances in a plurality of frame images before the current frame image, each of the image moving distances being a moving distance of the target vehicle in the image in each pair of adjacent frames. Determine whether the number of image moving distances greater than the distance threshold is greater than a set threshold. Based on the number of image moving distances greater than the distance threshold being greater than the set threshold, calculate the speed of the target vehicle in the current frame; based on the number of image moving distances greater than the distance threshold being less than or equal to the set threshold, track the driving trajectory of the target vehicle.

[0028] In some embodiments, the processor is further configured to perform the following steps: determine whether the speed of the target vehicle in each of the L consecutive frame images is outside a driving speed limit range of a road on which the target vehicle travels. Based on the speed of the target vehicle in each of the L consecutive frame images being outside the driving speed limit range of the road on which the target vehicle travels, perform a warning operation. Based on the speed of the target vehicle in at least one of the L consecutive frame images being within the driving speed limit range of the road on which the target vehicle travels, do not perform the warning operation. When the target vehicle triggers a warning in each of a plurality of speed detections in the monitoring video, increase the warning intensity when the target vehicle triggers the warning again in a speed detection.

[0029] In some embodiments, the processor is further configured to perform the following steps: obtaining, in the monitoring video of the road, motion trajectories of the plurality of target vehicles; determining whether motion trajectories of at least two adjacent target vehicles among the plurality of target vehicles are the same motion trajectory; if the motion trajectories of the two adjacent target vehicles are the same motion trajectory, detecting a speed of a current frame of the plurality of frames of images of the two adjacent target vehicles, and determining whether the speed of the rear target vehicle among the two adjacent target vehicles is continuously greater than the speed of the front target vehicle within a preset duration. Based on the speed of the rear target vehicle among the two adjacent target vehicles being continuously greater than the speed of the front target vehicle within the preset duration, performing a collision warning operation; and based on the speed of the rear target vehicle among the two adjacent target vehicles being less than or equal to the speed of the front target vehicle within the preset duration, not performing the collision warning operation.

[0030] In some embodiments, if the motion trajectories of the W adjacent target vehicles are the same motion trajectory, and W is greater than or equal to 3, the processor is further configured to perform the following steps: obtaining vehicle models of the W adjacent target vehicles, determining whether at least one small or medium-sized vehicle is located between two large-sized vehicles among the W adjacent target vehicles. Based on the presence of at least one small or medium-sized vehicle located between the two large-sized vehicles, performing a collision warning operation; and based on the absence of at least one small or medium-sized vehicle located between the two large-sized vehicles, detecting a speed of each frame of the plurality of frames of images of the W adjacent target vehicles, and determining whether the speed of the rear target vehicle among the W adjacent target vehicles is continuously greater than the speed of the front target vehicle within a preset duration. Based on the speed of the rear target vehicle among the W adjacent target vehicles being continuously greater than the speed of the front target vehicle within the preset duration, performing a collision warning operation; and based on the speed of the rear target vehicle among the W adjacent target vehicles being less than or equal to the speed of the front target vehicle within the preset duration, not performing the collision warning operation.

[0031] In another aspect, a vehicle warning system is provided, comprising: the electronic device in the above-mentioned aspect. A plurality of image acquisition devices electrically connected to the electronic device, the plurality of image acquisition devices being installed near a road, the plurality of image acquisition devices being used to shoot monitoring videos of the road and upload data of the monitoring videos to the electronic device. The plurality of image acquisition devices are electrically connected to the processor in the electronic device, and the processor stores the received monitoring videos in the memory in the electronic device.

[0032] In still another aspect, a non-transitory computer-readable storage medium is provided, comprising a computer program product stored on the non-transitory computer-readable storage medium; the computer program product comprising computer program instructions, which, when executed on a computer, cause the computer to perform the vehicle speed detection method according to any one of the embodiments of the first aspect and perform the vehicle collision warning method according to any one of the embodiments of the second aspect.

[0033] In still another aspect, a computer program product is provided. The computer program product comprises computer program instructions, which, when executed on a computer (e.g., a display device, a terminal device), cause the computer to perform the vehicle speed detection method according to any one of the embodiments of the first aspect and perform the vehicle collision warning method according to any one of the embodiments of the second aspect.

[0034] In still another aspect, a computer program is provided. When the computer program is executed on a computer (e.g., a display device, a terminal device), the computer program causes the computer to perform the vehicle speed detection method according to any one of the embodiments of the first aspect and perform the vehicle collision warning method according to any one of the embodiments of the second aspect. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the present disclosure, the following will briefly introduce the drawings needed to be used in some embodiments of the present disclosure. Obviously, the drawings described in the following description are only some drawings of the embodiments of the present disclosure, and other drawings can also be obtained by those skilled in the art according to these drawings. In addition, the drawings in the following description can be regarded as schematic diagrams, and are not limited to the actual size, actual process, actual time sequence, etc. of the products, methods, signals, etc. involved in the embodiments of the present disclosure.

[0036] Figure 1 A first flowchart of the vehicle speed detection method according to some embodiments of the present disclosure;

[0037] Figure 2 A second flowchart of the vehicle speed detection method according to some embodiments of the present disclosure;

[0038] Figure 3 A third flowchart of the vehicle speed detection method according to some embodiments of the present disclosure;

[0039] Figure 4 A fourth flowchart of the vehicle speed detection method according to some embodiments of the present disclosure;

[0040] Figure 5 A fifth flowchart of the vehicle speed detection method according to some embodiments of the present disclosure;

[0041] Figure 6 Sixth flowchart of a vehicle speed detection method according to some embodiments of the present disclosure;

[0042] Figure 7 Seventh flowchart of a vehicle speed detection method according to some embodiments of the present disclosure;

[0043] Figure 8 Eighth flowchart of a vehicle speed detection method according to some embodiments of the present disclosure;

[0044] Figure 9 A flowchart of a method for establishing an identity code of a target vehicle according to some embodiments of the present disclosure;

[0045] Figure 10 Ninth flowchart of a vehicle speed detection method according to some embodiments of the present disclosure;

[0046] Figure 11 Tenth flowchart of a vehicle speed detection method according to some embodiments of the present disclosure;

[0047] Figure 12 State diagram of a target detection frame overlapping of a target vehicle according to some embodiments of the present disclosure;

[0048] Figure 13 Eleventh flowchart of a vehicle speed detection method according to some embodiments of the present disclosure;

[0049] Figure 14 A flowchart of a method for calculating constraint parameters of an image acquisition device according to some embodiments of the present disclosure

[0050] Figure 15 Auxiliary diagram for calculating internal parameters of an image acquisition device in a frame of image according to some embodiments of the present disclosure;

[0051] Figure 16 Auxiliary diagram for calculating internal parameters of an image acquisition device according to some embodiments of the present disclosure;

[0052] Figure 17 Auxiliary diagram for calculating external parameters of an image acquisition device according to some embodiments of the present disclosure;

[0053] Figure 18 A flowchart of a vehicle collision warning method according to some embodiments of the present disclosure;

[0054] Figure 19 A frame of image for marking a detection region in a monitoring video according to some embodiments of the present disclosure;

[0055] Figure 20A flowchart for judging whether the motion trajectories of target vehicles are the same motion trajectory according to some embodiments of the present disclosure;

[0056] Figure 21 A structural diagram of an electronic device according to some embodiments of the present disclosure;

[0057] Figure 22 A structural diagram of a vehicle warning system according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0058] The technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings. Obviously, the described embodiments are only some of the embodiments of the present disclosure, but not all of the embodiments. Based on the embodiments provided by the present disclosure, all other embodiments obtained by a person of ordinary skill in the art belong to the scope of protection of the present disclosure.

[0059] Unless otherwise required by context, the term "comprise" and other forms of the term "comprise", such as "comprises" and "comprising", and the like, are used in an open, inclusive and non-limiting sense, that is, as "including, but not limited to". In the description of the specification, the terms "one embodiment", "some embodiments", "exemplary embodiments", "example", "specific example" or "some examples" and the like are intended to mean that a specific feature, structure, material or characteristic described in connection with the embodiment or example includes in at least one embodiment or example of the present disclosure. The illustrative representation of the above terms does not necessarily mean the same embodiment or example. In addition, the specific features, structures, materials or characteristics described can be included in any one or more embodiments or examples in any appropriate manner.

[0060] Hereinafter, the terms "first" and "second" are used only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first" and "second" can explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise stated, the meaning of "a plurality of" is two or more.

[0061] The use of "adapted for" or "configured for" herein means open and inclusive language that does not exclude devices adapted for or configured for performing additional tasks or steps.

[0062] In addition, the use of "based on" means open and inclusive, as a process, step, calculation, or other action that is "based on" one or more stated conditions or values can in practice be based on additional conditions or values beyond those stated.

[0063] With the rapid development of economy, the number of vehicles owned by residents in China is increasing year by year, and traffic accidents have become one of the important factors threatening people's life safety. Speed limit for cars is one of the effective means to prevent accidents and serious accidents.

[0064] Currently, vehicle speed measurement generally uses radar speed measurement and interval speed measurement. Among them, radar speed measurement mainly uses the principle of Doppler Effect: when the target approaches the radar, the reflected signal frequency will be higher than the transmitter frequency; on the contrary, when the target moves away from the radar, the reflected signal frequency will be lower than the transmitter frequency. Radar speed measurement can detect the instantaneous speed of the vehicle. Interval speed measurement is to arrange two adjacent monitoring points on the same road section, and calculate the average driving speed of the vehicle on the road section based on the time of the vehicle passing through the two monitoring points.

[0065] Radar speed measurement can only detect the speed of the vehicle at the moment when the vehicle approaches the radar, and cannot measure the real-time speed of the vehicle on a long-distance road section; while interval speed measurement has a certain lag, and can only obtain the average speed of the vehicle on a certain road section, and cannot obtain the real-time speed of the vehicle on a certain road section.

[0066] Based on this, some embodiments of the present disclosure provide a vehicle speed detection method, as shown in Figure 1 and Figure 6 The vehicle speed detection method is based on video monitoring, and the method comprises S110-S160.

[0067] S110, acquiring a monitoring video of a road, and extracting a plurality of continuous frames of images in the monitoring video.

[0068] Exemplarily, the monitoring video of the road includes the monitoring video of the urban road, the monitoring video of the rural trunk road, and the monitoring video of the expressway. The monitoring video is frame processed by using a video processing software, for example, Open-cv software, each frame of image of the monitoring video is extracted, and a plurality of continuous frames of images of the monitoring video are acquired. Figure 15 and Figure 19 For a frame of image in the plurality of frames of images of the monitoring video of the road, it can be seen that the frame of image includes a road and a plurality of vehicles.

[0069] S120, identifying the vehicles in the plurality of frames of images, and establishing a driving track of a target vehicle.

[0070] In some examples, as Figure 19As shown, the target detection and tracking algorithm is used to detect multiple frames of images, determine the target vehicle and establish a target detection box Bx (bounding box, bbox), track the target vehicle based on the target detection box Bx, and obtain the motion trajectory of the target vehicle.

[0071] It should be noted that the target vehicle in the monitoring video can be one or multiple. In the case of multiple target vehicles, a corresponding target detection box Bx is established for each target vehicle, and a motion trajectory corresponding to each target vehicle is formed. The tracking algorithm is used to track multiple target vehicles.

[0072] Exemplarily, the target detection algorithm model adopts a yolov5 algorithm, which is used to identify the target vehicle in the multiple frames of images extracted in S110, and mark the target vehicle with a target detection box Bx. The tracking algorithm model adopts a sort algorithm, which cooperates with the yolov5 algorithm. Among them, the yolov5 algorithm provides detection targets, such as target vehicles, and the sort algorithm realizes tracking of the target vehicles in multiple frames of images.

[0073] S130, obtain the image position coordinates of the target vehicle in each frame of image.

[0074] Exemplarily, an image coordinate system is established on each frame of image, and the origin of the image coordinate system coincides with the center of each frame of image. Then, the computer obtains the image position coordinates of the target vehicle in the corresponding image coordinate system according to the image coordinate position provided by the target detection box data obtained by the yolov5 algorithm in S120. For example, as shown in Figure 19 As shown, the image position coordinates of the target detection box in the image are the image position coordinates of the corresponding target vehicle.

[0075] It should be noted that the image position coordinates of the target detection box are the position coordinates of the center point Bo of the target detection box.

[0076] S140, obtain the world position coordinates of the target vehicle in the real world according to the image position coordinates.

[0077] For each image, the image position coordinates in the image and the real-world world position coordinates corresponding to the image have a conversion relationship. In some examples, for any one of the plurality of images obtained by S110, a parameter matrix of an image acquisition device for shooting the monitoring video is pre-calculated, for example, the image acquisition device can be a camera or a camera, and the parameter matrix of the image acquisition device can reflect the conversion relationship between the image position coordinates in the image and the real-world world position coordinates corresponding to the image. Through the parameter matrix of the image acquisition device, the image position coordinates of a certain point in the plurality of images can be converted into the world position coordinates of the point in the real world.

[0078] Exemplarily, a world coordinate system in the real world is established, wherein the origin of the world coordinate system can be set according to the situation, and generally, to reduce the amount of calculation, the X axis of the world coordinate system is parallel to the X axis of the image coordinate system, and the Y axis of the world coordinate system is parallel to the Y axis of the image coordinate system. According to the image position coordinates of the target vehicle (the image position coordinates of the target detection frame center point Bo of the target vehicle) in each image obtained by S130, the corresponding world coordinate position of the target vehicle in the world coordinate system in each image is calculated through the pre-calculated parameter matrix.

[0079] S150, according to the world position coordinates, the moving distance Ly of the target vehicle in the real world in each adjacent two images in the plurality of images is calculated. Exemplarily, according to the world coordinate position of the target vehicle in each image obtained by S140, a vector of the same target vehicle in the world coordinate system in the adjacent two images is established, and the length of the vector is the moving distance Ly of the target vehicle in the real world.

[0080] S160, according to the driving distance L of the target vehicle in the real world and the frame rate of the monitoring video, the speed Vd of the target vehicle in the current frame is calculated, wherein the driving distance L is obtained according to the moving distance Ly.

[0081] In some embodiments, the method for calculating the speed Vd of the target vehicle in the current frame according to the driving distance L of the target vehicle in the real world and the frame rate of the monitoring video in S160 comprises:

[0082] S161, the sum of the moving distance Ly of the target vehicle in the real world in each adjacent two images in the current frame image and the N frame images before the current frame image is calculated, and the sum of the moving distance Ly is taken as the driving distance L.

[0083] S162, according to the frame rate of the monitoring video, the time difference Tt between the current frame image and the Nth frame image before the current frame image is obtained.

[0084] S163、According to the driving distance and the time difference, a calculated speed of the target vehicle in the current frame is obtained.

[0085] For example, according to the movement distance Ly of the target vehicle in the real world in each two adjacent frames of images obtained in S150, the driving distance L of the target vehicle in the real world is the sum of the movement distance Ly of the target vehicle in the real world in each two adjacent frames of images in the current frame image and N frame images before the current frame image, that is, the driving distance L = movement distance Ly1 + movement distance Ly2… + movement distance LyN, wherein the movement distance Ly1 is the movement distance of the target vehicle in the real world in the current frame image and the first frame image before the current frame image, the movement distance Ly2 is the movement distance of the target vehicle in the real world in the first frame image before the current frame image and the second frame image before the current frame image, and the movement distance LyN is the movement distance of the target vehicle in the real world in the N-1 frame image before the current frame image and the N frame image before the current frame image. N For example, according to the movement distance Ly of the target vehicle in the real world in each two adjacent frames of images obtained in S150, the driving distance L of the target vehicle in the real world is the sum of the movement distance Ly of the target vehicle in the real world in each two adjacent frames of images in the current frame image and N frame images before the current frame image, that is, the driving distance L = movement distance Ly1 + movement distance Ly2… + movement distance LyN, wherein the movement distance Ly1 is the movement distance of the target vehicle in the real world in the current frame image and the first frame image before the current frame image, the movement distance Ly2 is the movement distance of the target vehicle in the real world in the first frame image before the current frame image and the second frame image before the current frame image, and the movement distance LyN is the movement distance of the target vehicle in the real world in the N-1 frame image before the current frame image and the N frame image before the current frame image.

[0086] According to the frame rate of the monitoring video, the time t between two adjacent frames of images can be obtained, for example, the frame rate of the monitoring video is 10 fps, then the time t between two adjacent frames of images is 100 ms, and the time difference between the current frame image and the Nth frame image before the current frame image is N*100 ms.

[0087] Therefore, the calculated speed Vs of the target vehicle in the current frame is the average speed of the driving distance L of the target vehicle in the time of the driving distance. Wherein, the time of the driving distance L is the time t between N adjacent two frames of images. That is:

[0088]

[0089] Wherein, 8≤N≤12, for example, N can be 8, 10 or 12.

[0090] For example, taking N as 10 and the current frame as the 100th frame, the sum L of N movement distances of the target vehicle in the real world before the 100th frame image is, from the (100-N)th frame, that is, from the 90th frame, the movement distance Ly1 of the target vehicle in the real world in the 90th frame image and the 91th frame image is calculated, the movement distance Ly2 of the target vehicle in the real world in the 91th frame image and the 92th frame image is calculated, and so on, the movement distance LyN of the target vehicle in the real world in the 99th frame image and the 100th frame image is calculated. 10 The movement distance Ly1~the movement distance Ly 10The sum of the ten moving distances Ly of the target vehicle before the 100th frame is the running distance L.

[0091] The calculated speed Vs of the target vehicle at the current frame of the 100th frame is the ratio of the sum L of the ten moving distances obtained above to the time difference (10x100ms) between the 90th frame image and the 100th frame image.

[0092] In some embodiments, the calculated speed Vs of the target vehicle at the current frame is taken as the speed Vd of the target vehicle at the current frame. That is, the speed Vd of the current frame = the calculated speed Vs of the current frame, for example, the calculated speed Vs of the target vehicle at the 100th frame is taken as the speed Vd of the target vehicle at the 100th frame.

[0093] In other embodiments, the calculated speed Vs of the target vehicle at the current frame and the calculated speed Vs of each frame image in the M frame images before the current frame image of the target vehicle are subjected to data smoothing processing, and the result of the data smoothing processing is taken as the speed Vd of the target vehicle at the current frame, where 3≤M≤5, for example, M can be 3, 4 or 5.

[0094] For example, taking the calculated speeds Vs of the same target vehicle at multiple current frames, with M being 3, and the speed Vd of the current frame of the target vehicle being the speed of the 100th frame of the target vehicle as an example, the M frame images before the current frame image of the target vehicle are the 99th frame image, the 98th frame image and the 97th frame image, and the calculated speeds Vs of the same target vehicle at multiple current frames include the calculated speed Vs of the target vehicle at the 100th frame 100 , the calculated speed Vs of the target vehicle at the 99th frame 99 , the calculated speed Vs of the target vehicle at the 98th frame 98 , and the calculated speed Vs of the target vehicle at the 97th frame 97 The calculated speed Vs of each frame image in the M frame images before the current frame image of the target vehicle is calculated using the method mentioned above, for example, taking N as 10 as an example, the calculated speed Vs of the target vehicle at the 99th frame 99 is the ratio of the sum of the moving distances Ly of the target vehicle in the real world in each adjacent two frame images from the 89th frame to the 99th frame to the time difference (10x100ms) between the 89th frame image and the 99th frame image.

[0095] The above four speeds are subjected to data smoothing processing, for example, the average of the above four speeds is taken as the current frame speed Vd of the 100th frame image of the target vehicle 100 , that is:

[0096]

[0097] In other examples, the disclosure adopts a moving average method to data smooth the current frame speed Vs, according to the formula:

[0098]

[0099] Wherein, the current frame is the mth frame, Vd m is the speed of the target vehicle in the mth frame, Vs m-i is the calculated speed of the target vehicle in the m-i frame, Vs m+i is the calculated speed of the target vehicle in the m+i frame, Vs m is the calculated speed of the target vehicle in the mth frame. For example, when M is 1, the current frame speed Vd 100 of the target vehicle in the 100th frame image is the average value of the current frame calculated speed Vs 99 of the target vehicle in the 99th frame image to the current frame calculated speed Vs 101 of the target vehicle in the 101th frame image.

[0100] By data smoothing the current frame calculated speed Vs of the target vehicle in each frame of the collected image, for example, taking the average value of multiple current frame calculated speeds Vs as the final obtained current frame speed Vd of the target vehicle, the problem of large error of a current frame calculated speed Vs caused by the target detection box jitter and the speed change of the target vehicle can be avoided, the accuracy of the obtained current frame speed Vd of the target vehicle can be improved, and the large interference of a single large error previous frame calculated speed Vs on the subsequent warning of judging the overspeed or low-speed driving of the target vehicle can be avoided.

[0101] The vehicle speed detection method provided by the disclosure can continuously detect the target vehicle on the road through the camera or the camera, convert the image position coordinates of the target vehicle on the image into the world position coordinates in the real world by obtaining the image position coordinates of the target vehicle on the image, realize the calculation of the moving distance of the target vehicle in the real world and the instantaneous speed of the target vehicle on a section of road, apply the computer vision technology to the vehicle speed detection method, realize the real-time detection of the vehicle speed, and can detect the speed of the vehicle in a specific frame, improve the real-time and accuracy of the vehicle speed detection. The vehicle speed detection method can take into account the advantages of the existing interval speed measurement and radar speed measurement on one hand, for example, the speed of a section of road can be measured as a whole, and at the same time the instantaneous speed of a section of road can be obtained. It can realize the supplement or assistance of the existing target vehicle speed measurement; on the other hand, the existing road monitoring system and interval speed measurement system can be directly used, for example, the monitoring device of the existing interval speed measurement system is used, and the image captured by the monitoring and shooting of the interval speed measurement system is processed by the method of the above embodiment. Or using the existing monitoring system, such as the sky eye system, etc. The speed of the target vehicle can also be obtained, without the need for large-scale investment of funds.

[0102] In some embodiments, as shown in Figure 2 and Figure 6 Before calculating the moving distance Ly of the target vehicle in the real world, the vehicle speed detection method further comprises S141 and S142.

[0103] S141, calculating the moving distance Lp of the target vehicle in the image in the adjacent two images in the plurality of images.

[0104] S142, judging whether the moving distance Lp of the target vehicle in the image is greater than the distance threshold Q.

[0105] If yes, the moving distance Ly of the target vehicle in the real world is calculated, that is, S150 is executed; if no, the driving trajectory tracking of the target vehicle is continued.

[0106] That is, based on the moving distance Lp of the target vehicle in the image being greater than the distance threshold Q, the moving distance Ly of the target vehicle in the real world is calculated; based on the moving distance Lp of the target vehicle in the image being less than or equal to the distance threshold Q, the driving trajectory tracking of the target vehicle is continued.

[0107] Exemplarily, after obtaining the image position coordinates of the target vehicle in each image, a vector of the same target vehicle in the image coordinate system in the adjacent two images is established, and the length of the vector in the image coordinate system is calculated. The length of the vector in the image coordinate system is the moving distance Lp of the target vehicle in the adjacent two images. The moving distance Lp of the target vehicle in the adjacent two images is compared with the distance threshold Q. When the moving distance Lp of the target vehicle in the adjacent two images is greater than the distance threshold Q, S150 is performed to obtain the current frame speed Vd of the target vehicle; when the displacement distance Lp is less than or equal to the distance threshold Q, S120 is returned to perform target detection on the target vehicle.

[0108] Exemplarily, the distance threshold Q is 1 / 10 of the length of the detection box Bx of the target vehicle.

[0109] By calculating the moving distance Lp of the target vehicle in each image and comparing it with the distance threshold Q, the target vehicle with extremely low moving distance in the real world and extremely low running speed of the target vehicle can be excluded. The speed of the target vehicle is approximately zero, and there is no calculation significance. This can reduce the occupation of the calculation amount of the hardware device by such target vehicles, thereby improving the running efficiency of the speed measurement system.

[0110] In some embodiments, as shown in Figure 3 and Figure 6 Before calculating the current frame speed Vd of the target vehicle, the vehicle speed detection method further comprises S151 and S152.

[0111] S151, calculate the number of image moving distances greater than the distance threshold Q in the multiple frames of images before the current frame image; wherein each image moving distance is the moving distance of the target vehicle in the image between each two adjacent frames of images.

[0112] S152, determine whether the number of image moving distances greater than the distance threshold Q is greater than a set threshold X. If yes, proceed to S160 to obtain the current frame speed Vd of the target vehicle, and if no, return to S120 to perform target detection on the target vehicle.

[0113] That is, based on the number of image moving distances greater than the distance threshold Q being greater than the set threshold X, the speed of the target vehicle in the current frame is calculated; and based on the number of image moving distances greater than the distance threshold Q being less than or equal to the set threshold X, the driving trajectory of the target vehicle is tracked.

[0114] In the above embodiment, the image moving distance is defined as the moving distance Lp of the target vehicle in the image between each two adjacent frames of images, for example, there are M1 frames of images before the current frame image, and the target vehicle has traveled M1 image moving distances to the current frame. The number of image moving distances (the moving distance Lp of the target vehicle in the image between each two adjacent frames of images) greater than the distance threshold Q is obtained, for example, the number is M2. In the monitoring video, if the number M2 is greater than the set threshold X, i.e., the target vehicle runs a sufficient distance in the field of view of the image acquisition device, the calculated speed Vs of the target vehicle in the current frame can be calculated. If the number M2 is less than or equal to the set threshold X, i.e., the target vehicle does not run a sufficient distance in the field of view of the image acquisition device, the calculated speed Vs of the target vehicle in the current frame is not calculated at this time, and the vehicle in the multiple frames of images is identified to establish the driving trajectory of the target vehicle.

[0115] Exemplarily, the set threshold X is 9-11, for example, the set threshold can be 9, 10 or 11.

[0116] In some embodiments, as shown in Figure 4 and Figure 6 the vehicle speed detection method further comprises S170.

[0117] S170, determine whether the speed of the target vehicle in each frame of the continuous L frames of images is outside the driving speed limit range of the driving road of the target vehicle. 3≤L≤7, for example: L can be 3, 5 or 7.

[0118] If yes, perform S180 to perform a warning operation. If no, do not perform a warning operation. Exemplarily, the driving trajectory of the target vehicle is continuously tracked

[0119] That is, based on the speed of the target vehicle in each frame of the continuous L frame images being outside the speed limit range of the road on which the target vehicle is driving, a warning operation is performed; and based on the speed of the target vehicle in at least one frame of the continuous L frame images being within the speed limit range of the road on which the target vehicle is driving, no warning operation is performed.

[0120] As shown in Figure 5 or Figure 6 The target vehicle speed detection side method further includes S180'.

[0121] When S180', the target vehicle triggers a warning when performing speed detection in multiple monitoring videos, and the target vehicle triggers a warning when performing speed detection again, the warning intensity is increased.

[0122] Exemplarily, L can be 5. After obtaining the current frame speed Vd of the target vehicle, when the continuous 5 current frame speeds Vd are all higher or lower than the speed limit range of the road on which the target vehicle is driving, for example, the speed limit range of the urban road can be 0-40Km / h, 0-60Km / h or 0-80Km / h, and the speed limit range of the highway can be 100Km / h-120Km / h or 80Km / h-100Km / h, indicating that the target vehicle is driving at an excessive speed or at a low speed, a warning operation is performed, indicating that the target vehicle is not driving normally. When the speed of the target vehicle on the road is within the speed limit range, or part of the current frame speed Vd is outside the speed limit range of the road on which the target vehicle is driving, but the number of continuous frames is less than 5, it indicates that the target vehicle is driving normally, and no warning operation is needed. The driving track of the target vehicle can be tracked.

[0123] When the target vehicle triggers a warning in the monitoring videos of multiple image collection devices or in multiple monitoring videos of the same image collection device, for example, the target vehicle triggers a warning in the monitoring videos of two image collection devices or in two monitoring videos of the same image collection device, and the target vehicle triggers a warning again, the intensity of this warning can be increased.

[0124] Triggering a warning or performing a warning operation can be a hardware device running the vehicle speed detection method provided by the present disclosure. The position information of the illegal vehicle in the image, the screenshot information, etc. are transmitted to the data center, and the data center decides the form of the warning sent to the user end. The specific form can be designed according to the customization requirements of the user end. The user end can be a traffic management platform system, which can draw a warning picture and save a warning record for management personnel to use.

[0125] In some embodiments, as Figure 7As shown, before identifying the vehicle in the multiple frames of images and establishing the driving track of the target vehicle, the vehicle speed detection method further includes S111.

[0126] S111, as Figure 19 As shown, the detection area QE is marked on the multiple frames of images, wherein the detection area QE is a closed graph and is located in the driving area of the road in the image, and the boundary of the detection area QE has fixed image position coordinates on each frame of image.

[0127] According to the world position coordinates, the moving distance Ly of the target vehicle in the real world in the adjacent two frames of images in the multiple frames of images is calculated, including: according to the world position coordinates, the moving distance Ly of the target vehicle located in the detection area QE in the adjacent two frames of images in the multiple frames of images in the real world is calculated.

[0128] The speed Vd of the target vehicle in the current frame is obtained, including: the speed Vd of the target vehicle located in the detection area QE in the current frame is calculated.

[0129] Exemplarily, the detection area QE is a closed area, and the length and width of the detection area QE on the image are greater than or equal to 80 pixels. For example, the area corresponding to the detection area QE in the real world is located in the driving area of the road in the image, for example, the first lane, the second lane and the third lane of the road, and the target vehicle running in the detection area QE is the target vehicle to be detected for speed, and the target vehicle stopping in the peripheral part (for example, the parking space on one side or both sides of the lane) of the detection area QE does not belong to the target vehicle to be detected for speed.

[0130] The detection area QE is used to demarcate the speed detection range of the target vehicle. On the one hand, the target vehicle in the detection area QE is clearly photographed, and the speed detection accuracy is high. On the other hand, the parking spaces near the road or the yard are excluded, and the system calculation amount occupied by unnecessary speed detection vehicles is reduced.

[0131] In some embodiments, as Figure 8 As shown, after the driving track of the target vehicle is established, the vehicle speed detection method further includes S121.

[0132] S121, the identity information list of the target vehicle is established; as Figure 9 As shown, including:

[0133] S1211, a re-identification model is used to extract a feature vector of the target vehicle.

[0134] S1212, the cosine of the angle of the feature vector of the target vehicle in each adjacent two frames of images is calculated.

[0135] S1213, whether the value of the cosine of the angle is greater than the similarity threshold value continuously G times is judged.

[0136] If yes, S1214 is performed, and an identity information list of the target vehicle is established in a vehicle information retrieval library, and the feature vector of the target vehicle is stored in the identity information list of the target vehicle; wherein the identity information list of the target vehicle includes the identity information of the target vehicle; if no, the target vehicle is detected and tracked. Wherein 3≤G≤7, and the similarity threshold is 0.42-0.48.

[0137] That is, based on the value of the included angle cosine being greater than the similarity threshold for G consecutive times, an identity information list of the target vehicle is established in a target vehicle information retrieval library, and the feature vector of the target vehicle is stored in the identity information list of the target vehicle; wherein the identity information list of the target vehicle includes the identity information of the target vehicle.

[0138] In some examples, the re-identification model adopts a Mutiple Granularity Network (MGN) model of resnet50, which can extract global features and local features of the detected target, and improve the recognition accuracy.

[0139] Exemplarily, the feature of the target vehicle is extracted, and the feature vector of the target vehicle is obtained. A plurality of feature vectors in adjacent frame images are compared, and the included angle cosine of the feature vectors is calculated. If there is a value of the included angle cosine that is greater than the similarity threshold for G consecutive times, for example, G can be 3, 5 or 7; the similarity threshold is 0.42, 0.45 or 0.48; the identity information list of the target vehicle is established, and the latest feature vector is stored in the identity information list of the target vehicle. A plurality of identity information lists of the target vehicle are set in a vehicle information retrieval library. The vehicle information retrieval library is a database set in the system in advance, which includes a plurality of identity information lists of vehicles. Each identity information list of a vehicle includes information that can indicate the identity characteristics of the vehicle, for example, the identity information list includes: the latest vehicle feature vector, the vehicle appearance time, the vehicle movement direction, the shooting camera serial number, etc. If there is no value of the included angle cosine that is greater than the similarity threshold for G consecutive times, the target vehicle is continuously tracked and the feature vector is extracted.

[0140] The identity information list of the target vehicle is established, which facilitates the establishment of the feature information of the target vehicle, the detection speed in the continuous tracking process, etc. in the corresponding identity information list of the target vehicle, which is beneficial to the identity recognition and identification of a plurality of target vehicles, and can make each vehicle correspond to a tracking ID (Identity document). In the process of issuing a warning to the target vehicle, the target vehicle in the monitoring video corresponds to the target vehicle in the real world.

[0141] In some embodiments, the monitoring of the target vehicle in the video for target detection and tracking also includes some special cases, for example, the target vehicle tracking is lost, or the target detection boxes of two target vehicles overlap.

[0142] In the case of target vehicle tracking loss, for example, the number of target detection boxes Bx in the current frame image is reduced compared with the number of target detection boxes Bx in the previous frame image, or in the tracking process of the sort algorithm, the information of a certain tracking target suddenly disappears. As shown in Figure 10 In the case of target vehicle tracking loss, S122 is entered.

[0143] S122, determine whether the target detection box Bx of the target vehicle in the previous frame image in which the target vehicle tracking is lost is located within the detection area QE.

[0144] If yes, S124 is executed: match the feature vector of the target vehicle whose tracking is lost with the feature vector of the newly acquired target vehicle after the tracking loss, and establish the identity information of the newly acquired target vehicle which matches the feature vector in the identity information list of the target vehicle whose tracking is lost. If no, stop searching for the target vehicle.

[0145] That is, based on the target detection box Bx of the target vehicle being located within the detection area QE, the feature vector of the target vehicle whose tracking is lost is matched with the feature vector of the newly acquired target vehicle after the tracking loss, and the identity information of the newly acquired target vehicle which matches the feature vector is established in the identity information list of the target vehicle whose tracking is lost. Based on the target detection box Bx of the target vehicle being located outside the detection area QE, stop searching for the target vehicle.

[0146] In some examples, when the target vehicle tracking is lost, the feature vector of the newly appeared target vehicle can be matched with the feature vector of the lost target vehicle, the matching rule is to calculate the cosine value between the feature vector of the newly appeared target vehicle and the feature vector of the lost tracking target vehicle, and when the cosine value is greater than the similarity threshold value, the two vehicles are considered to be the same vehicle, the motion trajectory is merged, and the identity information list in the target vehicle information retrieval library is updated.

[0147] In addition, there is a case where the target vehicle drives out of the detection area QE, at this time the target detection box Bx of the target vehicle is not within the detection area QE, and the target vehicle which is not in the detection area QE does not need to be detected. There is also a case where the target vehicle drives out of the detection area QE and then quickly drives out of the field of view range of the image acquisition device, so when the previous frame image in which the target vehicle tracking is lost is not within the detection area QE, the target vehicle does not need to be searched.

[0148] In some embodiments, as shown in Figure 11 and Figure 19 shown, the detecting the multiple frames of images, determining the target vehicle and establishing a target detection box, tracking the target vehicle based on the target detection box, and obtaining the motion trajectory of the target vehicle, further comprises S123

[0149] S123, there are multiple target detection boxes Bx, in the case of more than two target detection boxes Bx overlapping, the feature vector of the target vehicle corresponding to the overlapping target detection boxes Bx is extracted. The target vehicles with the cosine value of the feature vector greater than the similarity threshold in adjacent two frames of images are established in the same identity information list.

[0150] In the case of target detection boxes Bx overlapping, the target tracking process is prone to cause the identity information list of the target vehicle corresponding to the target detection boxes Bx to be incorrectly exchanged. For example, as shown in Figure 12 , the A target detection box Bx and the B target detection box Bx in the figure represent two overlapping target detection boxes in the Jth frame of image, and the C target detection box Bx and the D target detection box Bx represent two overlapping target detection boxes in the J+1th frame of image. And the A target detection box Bx and the D target detection box Bx in the figure correspond to the same vehicle, and the B target detection box Bx and the C target detection box Bx correspond to the same vehicle. However, because of the overlapping state of the target detection boxes Bx, the tracking process is chaotic, which may cause the A target detection box Bx in the Jth frame of image and the C target detection box Bx in the J+1th frame of image to be recorded as the same target vehicle, and the B target detection box Bx in the Jth frame of image and the D target detection box Bx in the J+1th frame of image to be recorded as the same target vehicle. Therefore, the target vehicle feature extraction, as a supplement to the target detection and tracking process, can ensure the accurate association of the identity information list of the target vehicle with the corresponding target vehicle in the real world, and assist in supplementing when the tracking is lost and the target detection boxes overlap in the target detection and tracking process.

[0151] In some embodiments, as shown in Figure 13 , before obtaining the world position coordinates of the target vehicle in the real world according to the image position coordinates, S100 is further included.

[0152] S100, the internal parameters and external parameters of the image acquisition device used to shoot the monitoring video are calculated; the internal parameters and external parameters are used to convert the image position coordinates of the multiple frames of images and the corresponding world position coordinates. Wherein, the image acquisition device refers to a pinhole camera or a pinhole camera.

[0153] As shown in Figure 14As shown, S100 calculates the internal parameters and the external parameters of the image acquisition device for shooting the monitoring video, which includes:

[0154] S101, as Figure 15 As shown, the first vanishing point N1 and the second vanishing point N2 are labeled on the labeled image in the plurality of images. Among them, the labeled image is any frame in the plurality of images.

[0155] Exemplarily, at least two first vanishing lines XL1 and at least two second vanishing lines XL2 are labeled on the labeled image. One first vanishing line XL1 is parallel to one lane boundary, and the other first vanishing line XL1' is parallel to the other lane boundary. One second vanishing line XL2 is perpendicular to one lane boundary, and the other second vanishing line XL2' is perpendicular to the other lane boundary. Wherein, the intersection of two first vanishing lines XL1 is the first vanishing point N1, and the intersection of two second vanishing lines XL2 is the second vanishing point N2.

[0156] S102, the image position coordinates of the first vanishing point N1 and the second vanishing point N2 in the labeled image are obtained.

[0157] Exemplarily, as Figure 16 As shown, an image coordinate system is established on the labeled image. Wherein, the origin O of the image coordinate system coincides with the center of a certain frame of picture, and the coordinates of the first vanishing point N1 and the second vanishing point N2 are obtained.

[0158] S103, a straight line equation passing through the first vanishing point N1 and the second vanishing point N2 is established.

[0159] Exemplarily, a straight line equation ax+by+c=0 passing through the first vanishing point N1 and the second vanishing point N2 is established.

[0160] S104, the center of the labeled image is coincided with the principal point, and the initial internal parameters and the initial external parameters of the image acquisition device are calculated according to the straight line equation.

[0161] Exemplarily, the center O' of each labeled image coincides with the principal point. Wherein, the principal point is the intersection of the imaging plane (the plane of each labeled image) and the camera optical axis.

[0162] The initial internal parameters K of the image acquisition device are:

[0163]

[0164] Wherein, because the center O' of the plurality of pictures coincides with the principal point, i.e. u0 and v0 are both zero (u0 and v0 are the coordinates of the principal point on the image); the general model of the pinhole camera considers the tilt factor between the two image axes, denoted by γ, and the commonly used simplification method is to set the tilt to zero (γ = 0); in the labeled image, the projection point of the origin O on the straight line equation ax + by + c = 0 is No. In a certain frame of picture, the projection point of the origin O on the straight line equation ax + by + c = 0 is No. It is known by calculation that the distance between the origin O and the projection point No is ‖ONo‖, which is the focal length f.

[0165]

[0166] Wherein, Oc is the focal point of the image acquisition device, the straight line OcNo is perpendicular to the straight line equation ax + by + c = 0, i.e.

[0167]

[0168] The following is the initial external parameter calculation process:

[0169] The initial external parameters of the image acquisition device include a rotation matrix and a translation matrix. Wherein, R is the rotation matrix between the world coordinate system and the coordinate system of the camera; T is the translation vector between the world coordinate system and the coordinate system of the camera.

[0170] A vector relationship is established, and the vector relationship has the same direction as the world coordinate system, so the rotation between the coordinate system established by the vector relationship and the coordinate system of the camera is the same as the rotation between the world coordinate system and the coordinate system of the camera.

[0171] The vector relationship is: And Zz' = Xz' x Yz'. That is, the projection of Xz' on the labeled image is The projection of Yz' on the labeled image is In fact, the first vanishing line XL1 is parallel to the corresponding Xz1 axis of the Xz axis of the world coordinate system on the labeled image, and the second vanishing line XL2 is parallel to the corresponding Yz1 axis of the Yz axis of the world coordinate system on the labeled image.

[0172] In the Xz' direction, the direction Fx of the coordinate system established by the vector relationship is:

[0173]

[0174] In the Yz' direction, the direction Fy of the coordinate system established by the vector relationship is:

[0175]

[0176] In the direction of Xz' direction, the direction of coordinate system established by vector relationship is Fx:

[0177] Fz=FxFy

[0178] Wherein, N1x is The component vector in the direction of X axis of camera coordinate system, N1y is The component vector in the direction of Y axis of camera coordinate system. f is the focal length vector, the Z axis of camera coordinate system is on the camera optical axis, and the first vanishing point N1 and the second vanishing point N2 are on the marked image, so And The length of is the focal length f, and the direction is the positive direction of the Z axis of the camera coordinate system. N2x is the vector The component vector in the direction of X axis of camera coordinate system, N2y is the vector The component vector in the direction of Y axis of camera coordinate system, wherein the origin of the camera coordinate system is the camera focus point, i.e. Figure 16 The Oc point of the camera coordinate system, the X axis of the camera coordinate system is parallel to the Xt axis of the image coordinate system, the Y axis of the camera coordinate system is parallel to the Yt axis of the image coordinate system, the Xt axis of the image coordinate system can coincide with the first vanishing line XL1, and the Yt axis of the image coordinate system can coincide with the second vanishing line XL2.

[0179] The above process obtains the direction of the vector relationship.

[0180] That is, the rotation matrix R is:

[0181]

[0182] In the above formula, Zzx is the angle of deflection of the Zz axis of the world coordinate axis and the Zt axis of the camera coordinate system in the direction of the Xt axis, Zzy is the angle of deflection of the Zz axis of the world coordinate axis and the Zt axis of the camera coordinate system in the direction of the Yt axis, and Zzz is the angle of deflection of the Zz axis of the world coordinate axis and the Zt axis of the camera coordinate system in the direction of the Zt axis.

[0183] Obtain the length of an object in the field of view of the camera, for example: as shown in Figure 17 A lane line Ll is 6m long, one end of the lane line Ll is located at the origin of the world coordinate system, so the coordinate positions of the two end points of the lane line Ll in the world coordinate system are P1=[0,0,0] T And P2=[P2x,P2y,P2z] T .

[0184] According to the known rotation matrix R, the coordinate positions of the two end points of the lane line Ll in the image coordinate system are P1' and P2', and have the following relationship:

[0185]

[0186] Wherein, the origin and the principal point of the image coordinate system coincide, so that one end (P1') of the line segment P1'P2' in the image coordinate system is translated to the origin of the image coordinate system, and the intersection of the line segment P1'P2' and OcP2 is Q, the coordinates of Q are obtained, and the length of P1'Q is calculated.

[0187] Because the triangle P1OcP2 and the triangle P1'OcQ are similar, therefore,

[0188]

[0189] The initial internal parameter K and the initial external parameter are obtained through the above calculation process, wherein the initial external parameter includes a rotation matrix R and a translation matrix T, and because there are stability problems of the image acquisition device in the process of obtaining the monitoring video, and problems of accuracy of the coordinates, the initial internal parameter K and the initial external parameter have large errors.

[0190] S105, selecting at least one calibration reference Cr on a certain frame of image, the calibration reference being a marker with a known distance between two ends in the real world, and the calibration reference including one line segment Cr1 of a dashed lane line, an interval line Cr2 between adjacent dashed lane lines, and an interval line Cr3 between two line segments connected in the same dashed lane line.

[0191] Exemplarily, the number of the calibration references Cr is 8-10, for example, the number of the calibration references Cr can be 8, 9 or 10, and taking the number of the calibration references Cr as 9 as an example, the distance between two ends of the nine calibration references Cr is known.

[0192] S106, obtaining the image position coordinates of the two end points of the calibration reference in the image.

[0193] For example, taking one line segment Cr1 of a dashed lane line as an example, the image position coordinates of the two end points of a calibration reference are P K ' or Q K '.

[0194] S107, taking the at least one calibration reference as a constraint condition, constructing a constraint formula, and iteratively calculating the initial internal parameter and the initial external parameter according to the constraint formula, and obtaining the internal parameter and the external parameter of the image acquisition device according to the optimal solution of the constraint formula.

[0195] Wherein, the constraint formula is:

[0196]

[0197] Wherein, N is the number of the calibration references Cr, Q is the world position coordinate of one end of the kth calibration reference Cr in the real world K is the world position coordinate of the other end of the kth calibration reference Cr in the real world is the image position coordinate of one end of the kth calibration reference Cr in the annotated image, the world position coordinate calculated by using the initial intrinsic parameters and the initial extrinsic parameters is the image position coordinate of the other end of the kth calibration reference Cr in the annotated image, the world position coordinate calculated by using the initial intrinsic parameters and the initial extrinsic parameters, and cp represents the constraint parameters of the image acquisition device, including the intrinsic parameters and the extrinsic parameters.

[0198] According to the image position coordinates P K ’ or Q K ’ of the two ends of the calibration reference obtained in S106, the world position coordinates of the calibration parameters Cr are calculated by the formula:

[0199]

[0200]

[0201] In some examples, the number of calibration parameters Cr is 9, that is, the following are obtained

[0202] ‖P K -Q K ‖2 is the known distance H of the two ends of the calibration reference Cr in the real world, so H1, H2, H3, H4, H5, H6, H7, H8 and H9 can be obtained.

[0203] B is the calculated distance of the two ends of the calibration reference Cr in the world coordinate system, so B1, B2, B3, B4, B5, B6, B7, B8 and B9 can be obtained.

[0204] The above values are brought into the constraint formula, the difference between the known distance H and the calculated distance B is summed for multiple pairs, and the initial intrinsic parameters and the initial extrinsic parameters are iterated. For example, the process of iterating the initial intrinsic parameters and the initial extrinsic parameters according to the constraint formula in S107 is as follows: the constraint formula is partially differentiated, and then the gradient descent method is used to iteratively update the constraint parameters, wherein the number of times of adjusting or updating in the process of iteratively updating the constraint parameters by using the gradient descent method can be 100 times, and the amplitude of each adjustment or update can be 0.01.

[0205] ​​​Due to the lack of correlation between the world coordinate system and the image coordinate system, the initial internal parameter K and the initial external parameter calculated by S104 have large errors in different shooting scenes. Therefore, by adding a constraint condition in the camera calibration process through S105-S107, the constraint parameter cp obtained is the relatively accurate internal parameter K of the image acquisition device, the rotation matrix R of the external parameter and the translation matrix T of the external parameter, which can improve the accuracy of the conversion of the image position coordinates of multiple frames of images and their corresponding world position coordinates. In addition, the calibration reference of the present disclosure adopts dashed lane lines Ll (for example, the length of one segment Cr1 of the dashed lane line is 6 meters, and the length of the interval line Cr2 between the dashed lane lines is 8 meters), the interval line Cr3 between the two connected segments in the same dashed lane line, the width line Cr4 and the length line Cr5 of the target vehicle, etc. The distance of the calibration reference is a known value in the real world, and there is no need for staff to measure on site, which improves efficiency and saves manpower.

[0206] The present disclosure also provides a car crash warning method, which comprises using the vehicle speed detection method provided in the above embodiments, acquiring a monitoring video of a road and extracting a plurality of continuous frames of images in the monitoring video, identifying vehicles in the plurality of frames of images, establishing a driving track of a target vehicle, and detecting the speed of the target vehicle.

[0207] As shown in Figure 18 and Figure 19 , the car crash warning method further comprises S210, S220, S230 and S230'.

[0208] S210, in the monitoring video of the road, the motion trajectories of a plurality of target vehicles are established.

[0209] S220, determining whether the motion trajectories of at least two adjacent target vehicles in the plurality of target vehicles are the same motion trajectory.

[0210] The determination result of S220 includes three kinds, the first kind is that the motion trajectories of two adjacent target vehicles in the plurality of target vehicles are the same motion trajectory, the second kind is that the motion trajectories of more than two adjacent target vehicles in the plurality of target vehicles are the same motion trajectory, and the second kind is that there is no motion trajectory of more than two adjacent target vehicles in the plurality of target vehicles is the same motion trajectory, and the present disclosure discusses the first kind and the second kind.

[0211] If the motion trajectories of the two adjacent target vehicles are the same motion trajectory, S230 and S230' are executed in turn:

[0212] S230, detecting the speed of each frame of the two adjacent target vehicles in the plurality of frames of images,

[0213] S230', determining whether the speed of the rear target vehicle is greater than the speed of the front target vehicle in the preset duration.

[0214] If yes, performing a collision warning operation, if not, not performing a collision warning operation, and returning to S210 to establish the movement trajectory of the plurality of target vehicles in the monitoring video of the road.

[0215] That is, based on the speed of the rear target vehicle being greater than the speed of the front target vehicle in the preset duration, the collision warning operation is performed. Based on the speed of the rear target vehicle being less than or equal to the speed of the front target vehicle in the preset duration, the collision warning operation is not performed, and the speed detection of the two target vehicles in the same movement trajectory can continue.

[0216] Triggering the collision warning or performing the collision warning can be a hardware device running the collision warning method provided by the present disclosure, and the position information of the illegal vehicle in the image, the screenshot information, etc. are transmitted to the data center, and the data center decides the form of the alarm sent to the user end. The specific form can be designed according to the customization requirements of the user end. The user end can be a traffic management platform system, which can draw an alarm picture and save an alarm record for management personnel to use.

[0217] Among the adjacent two target vehicles, the rear target vehicle is the target vehicle with a shorter movement trajectory, and the front target vehicle is the target vehicle with a longer movement trajectory.

[0218] For example, as shown in Figure 19 The movement trajectory of the target vehicle C1 is Lc1, and the movement trajectory of the target vehicle C2 is Lc2, wherein the length of Lc1 is greater than the length of Lc2, that is, the target vehicle C1 is determined as the front vehicle, and the target vehicle C2 is determined as the rear vehicle. The collision warning method can match the movement trajectories of the target vehicles entering the image capture device shooting area in the same time range, compare the driving speeds of the target vehicles on the same movement trajectory, and determine whether there is a collision risk. It can improve the emergency response speed of the accident, improve the safety risk awareness of the vehicle owner, and reduce the probability of the accident.

[0219] As shown in Figure 18 If the movement trajectories of the adjacent W target vehicles are the same movement trajectory, wherein W is greater than or equal to 3, the collision warning method further comprises S240, S241 and S242.

[0220] S240, obtaining the vehicle type of the adjacent W target vehicles.

[0221] S241, judging whether the vehicle types of the adjacent W target vehicles are at least one small or medium-sized vehicle located between two large vehicles.

[0222] If yes, performing the operation of collision warning; if no, calculating the speeds of the adjacent W target vehicles.

[0223] That is, based on at least one small or medium-sized vehicle located between two large vehicles, performing the operation of collision warning; based on the absence of small or medium-sized vehicles between the two large vehicles, calculating the speeds of the adjacent W target vehicles.

[0224] S242, judging whether the speed of the rear target vehicle is greater than the speed of the front target vehicle in the adjacent W target vehicles within a preset duration.

[0225] If yes, performing the operation of collision warning; if no, not performing the operation of collision warning, for example, returning to S210 to establish the motion trajectories of the plurality of target vehicles in the monitoring video of the road. That is, based on the presence of the case that the speed of the rear target vehicle is greater than the speed of the front target vehicle in the adjacent W target vehicles within a preset duration, performing the operation of triggering collision warning; based on the presence of the case that the speed of the rear target vehicle is less than or equal to the speed of the front target vehicle in the adjacent W target vehicles within a preset duration, not performing the operation of collision warning.

[0226] Exemplarily, the method for judging the vehicle type of the target vehicle calculates the width of the target vehicle in the target detection frame Bx according to the image position coordinates of the two end points in the width direction of the target detection frame Bx, and calculates the two world position coordinates corresponding to the two image position coordinates, so as to directly obtain that the target vehicle belongs to a large vehicle or a small vehicle or a medium-sized vehicle through the width of the target vehicle. For example, the width of the calculated target vehicle is 1.5 meters to 1.8 meters, which is a small vehicle, the width of the calculated target vehicle is 1.8 meters to 2.0 meters, which is a medium-sized vehicle, and the width of the calculated target vehicle is more than 2.0 meters, which is a large vehicle.

[0227] In the process of calculating the vehicle type, only the real-world width of the target detection frame Bx and a fixed value need to be compared, which avoids training a target vehicle attribute model to distinguish the type of the target vehicle and reduces the computational load of the system.

[0228] Exemplarily, as Figure 19As shown, there are multiple target vehicles on the road, and multiple target vehicles are arranged in sequence from front to back on the same motion trajectory, which can be the same lane, for example, target vehicle C1 and target vehicle C2 are on the same lane. The vehicle type on the same motion trajectory is judged. If there are two large target vehicles in the middle of the small target vehicle or the medium target vehicle on the same motion trajectory, a collision warning can be issued. If the vehicle type on the same motion trajectory is a small car team or a medium car team or a mixed team of small and medium cars or a large car team or the like, the speed of the target vehicle is detected, and it is judged that the speed of the rear target vehicle is always greater than the speed of the front target vehicle within a predetermined duration, for example, the predetermined duration can be 30 seconds, 60 seconds or 120 seconds, and then the collision warning is executed.

[0229] It should be noted that the length of the motion trajectory of the target vehicle on the same motion trajectory at the same time is judged to judge the front target vehicle or the rear target vehicle. The method of judging the length of the motion trajectory of the target vehicle on the same motion trajectory at the same time can be the length between the image position coordinates of the target vehicle in the current frame image and the image position coordinates of the same target vehicle in the first frame image of the monitoring video.

[0230] In some embodiments, as shown, Figure 20 S220: A method for judging whether the motion trajectories of at least two target vehicles are the same motion trajectory, comprising S2201 and S2202.

[0231] S2201, obtain the image position coordinate set of each target vehicle in multiple frames of images, and perform linear equation fitting to obtain the motion linear equation of each target vehicle in the image coordinate system. The origin of the image coordinate system coincides with the center of each frame of image.

[0232] S2202, judge whether the difference between the slopes of at least two motion linear equations is less than the slope threshold value, and the difference between the intercepts of at least two motion linear equations is less than the intercept threshold value.

[0233] If yes, it is determined that the motion trajectories of the at least two target vehicles corresponding to the at least motion linear equations are the same motion trajectory, wherein the slope threshold value ranges from 3 to 7, and the intercept threshold value ranges from 15 to 25.

[0234] That is, based on the difference between the slopes of at least two motion linear equations being less than the slope threshold value, and the difference between the intercepts of the at least two motion linear equations being less than the intercept threshold value, it is determined that the motion trajectories of the at least two target vehicles corresponding to the at least two motion linear equations are the same motion trajectory.

[0235] Exemplarily, according to the images obtained in S110, the target vehicle in the monitoring video is detected and tracked through S120 to establish the motion trajectory of the target vehicle, and the identity information list of the target vehicle is established through S121. The image position coordinates of the same target vehicle in each frame of the monitoring video are obtained, and the motion trajectory of the target vehicle in the image coordinate system is established through the image position coordinates. The motion trajectory of each target vehicle is fitted with a straight line equation, for example, the least square method can be used for fitting to generate a straight line equation. When the difference between the slopes of any two of the plurality of straight line equations is less than the slope threshold value, and the difference between the intercepts of any two of the plurality of straight line equations is less than the intercept threshold value, it is determined that the two target vehicles corresponding to the two straight line equations are in the same motion trajectory. The slope threshold value can be 3, 4 or 7. The intercept threshold value can be 15, 20 or 25.

[0236] In some embodiments, after the same vehicle triggers the collision warning operation in the collision warning of multiple monitoring videos, the alarm intensity is increased when the target vehicle triggers the collision warning operation again in the collision warning.

[0237] Exemplarily, when the target vehicle triggers the collision warning in the monitoring video of multiple image acquisition devices or in multiple monitoring videos of the same image acquisition device, for example, the target vehicle triggers the collision warning in the monitoring video of two image acquisition devices or in two monitoring videos of the same image acquisition device, the alarm intensity can be increased after the target vehicle triggers the collision warning again.

[0238] The present disclosure also provides an electronic device 10, such as Figure 21 As shown, an electronic device 10 includes a processor 1 and a memory 2. The processor 1 is configured to perform the following steps: obtaining a monitoring video of a road and storing the monitoring video to the memory 2; extracting a plurality of consecutive frames of images in the monitoring video. The vehicles in the plurality of frames of images are identified to establish the driving trajectory of the target vehicle. The image position coordinates of the target vehicle in each frame of image are obtained. According to the image position coordinates, the world position coordinates of the target vehicle in the real world are obtained. The moving distance Ly of the target vehicle in the real world between two adjacent frames of images of the monitoring video of the road is calculated. According to the driving distance L of the target vehicle in the real world and the frame rate of the monitoring video, the speed Vd of the target vehicle in the current frame is calculated, wherein the driving distance L is obtained according to the moving distance Ly.

[0239] The processor 1 may, for example, be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic, discrete hardware components, or the like. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0240] The memory 2 can be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disk storage, a magnetic disk storage or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited thereto. The memory 2 can exist independently and be connected to the processor via a communication bus. The memory 2 can also be integrated with the processor 1.

[0241] The implementation of the computer program stored on the memory 2 and executable on the processor 1, such as the implementation of processes or functions, can be implemented with separate software modules that allow at least one function or operation to be performed. The software code can be implemented with any appropriate programming language written software application (or program) and can be stored in the memory and executed by the processor unit.

[0242] An electronic device 10 has a function of performing the target vehicle speed detection in the above-described embodiment of one aspect, and has the same advantageous effects as the above-described vehicle speed detection method.

[0243] In some embodiments, the processor 1 is further configured to perform the following steps: calculating a sum of moving distances Ly of the target vehicle in the real world in each of two adjacent frames of the monitoring video of the road, the sum of the moving distances Ly being the driving distance L. According to the frame rate of the monitoring video, a time difference between the current frame and the Nth frame before the current frame is obtained. According to the driving distance L and the time difference Tt, a calculated speed Vs of the target vehicle in the current frame is obtained, and the calculated speed Vs of the target vehicle in the current frame is taken as the speed Vd of the target vehicle in the current frame.

[0244] In other embodiments, the processor 1 is further configured to perform the following steps: calculating the speed of the target vehicle in the current frame according to the driving distance L of the target vehicle in the real world and the frame rate of the monitoring video, including: calculating a sum of moving distances Ly of the target vehicle in the real world in each of two adjacent frames of the monitoring video of the road, the sum of the moving distances Ly being the driving distance L. According to the frame rate of the monitoring video, a time difference between the current frame and the Nth frame before the current frame is obtained. According to the driving distance L and the time difference, a calculated speed Vs of the target vehicle in the current frame is obtained; performing data smoothing on the calculated speed Vs of the target vehicle in the current frame and the calculated speed of each of the M frames before the current frame, and taking the result of the data smoothing as the speed Vd of the target vehicle in the current frame.

[0245] In some embodiments, before the processor 1 is configured to perform the calculation of the moving distance Ly of the target vehicle in the real world in each of two adjacent frames of the monitoring video of the road, the processor 1 is further configured to perform the following steps: calculating a moving distance Lp of the target vehicle in the image in each of two adjacent frames of the monitoring video of the road; judging whether the moving distance Lp in the image is greater than a distance threshold Q; based on the moving distance Lp of the target vehicle in the image being greater than the distance threshold Q, calculating the moving distance Ly of the target vehicle in the real world; based on the moving distance Lp of the target vehicle in the image being less than or equal to the distance threshold Q, continuing to track the driving trajectory of the target vehicle.

[0246] In some embodiments, before the processor 1 is configured to calculate the speed Vd of the target vehicle in the current frame, the processor 1 is further configured to perform the following steps: calculate the number of image movement distances greater than a distance threshold Q among a plurality of image movement distances in a plurality of frames of images before the current frame of image. Each image movement distance is the movement distance Lp of the target vehicle in the image between each adjacent two frames of images. Determine whether the number of image movement distances greater than the distance threshold Q is greater than a set threshold X. Based on the number of image movement distances greater than the distance threshold Q being greater than the set threshold X, calculate the speed of the target vehicle in the current frame; based on the number of image movement distances greater than the distance threshold Q being less than or equal to the set threshold X, perform driving trajectory tracking on the target vehicle.

[0247] In some embodiments, the processor 1 is further configured to perform the following steps: determine whether the speed of the target vehicle in each frame of the consecutive L frames of images is outside the driving speed limit range of the road on which the target vehicle travels. Based on the speed of the target vehicle in each frame of the consecutive L frames of images being outside the driving speed limit range of the road on which the target vehicle travels, perform a pre-warning operation; based on the speed of the target vehicle in at least one frame of the consecutive L frames of images being within the driving speed limit range of the road on which the target vehicle travels, do not perform the pre-warning operation. When the target vehicle triggers a pre-warning every time the speed of the target vehicle is detected in the plurality of monitoring videos, the warning intensity is increased when the target vehicle triggers a pre-warning again when the speed of the target vehicle is detected again.

[0248] In some embodiments, the processor 1 is further configured to perform the following steps: obtain a plurality of motion trajectories of a plurality of target vehicles in the monitoring video of the road. The processor 1 is further configured to determine whether the motion trajectories of at least two adjacent target vehicles among the plurality of target vehicles are the same. Based on the motion trajectories of the two adjacent target vehicles being the same, detect the speed of the current frame of the two adjacent target vehicles in a plurality of frames of images, and determine whether the speed of the rear target vehicle is continuously greater than the speed of the front target vehicle among the two adjacent target vehicles within a preset duration. Based on the speed of the rear target vehicle being continuously greater than the speed of the front target vehicle among the two adjacent target vehicles within the preset duration, perform a collision warning operation. Based on the speed of the rear target vehicle being less than or equal to the speed of the front target vehicle among the two adjacent target vehicles within the preset duration, do not perform the collision warning operation.

[0249] In some embodiments, if the motion trajectories of W adjacent target vehicles are the same, and W is greater than or equal to 3, then processor 1 is further configured to perform the following steps: obtain the vehicle models of the W adjacent target vehicles, and determine whether at least one small or medium-sized vehicle is located between two large vehicles. Based on the presence of at least one small or medium-sized vehicle between the two large vehicles, perform a collision warning operation. Based on the absence of at least one small or medium-sized vehicle between the two large vehicles, detect the speed of the W adjacent target vehicles in each frame of the multi-frame image, and determine whether, within a preset duration, the speed of the rear target vehicle among the W adjacent target vehicles is continuously greater than the speed of the front target vehicle. Based on the presence of a rear target vehicle among the W adjacent target vehicles whose speed is continuously greater than the speed of the front target vehicle within the preset duration, perform a collision warning operation. Based on the presence of a rear target vehicle among the W adjacent target vehicles whose speed is less than or equal to the speed of the front target vehicle within the preset duration, do not perform a collision warning operation.

[0250] This disclosure also provides a vehicle warning system 100, such as Figure 22 As shown, the vehicle warning system 100 includes: the electronic device 10 described in the above embodiment; a plurality of image acquisition devices 20 electrically connected to the electronic device 10; the plurality of image acquisition devices 20 being installed near the road; and the plurality of image acquisition devices 20 being used to capture surveillance video of the road and upload the surveillance video data to the electronic device 10. The plurality of image acquisition devices 20 are electrically connected to the processor 1 in the electronic device 10, and the processor 1 stores the received surveillance video in the memory 2 of the electronic device 10.

[0251] A vehicle warning system 100 employs the aforementioned electronic device 10 and has the same beneficial effects as the vehicle speed detection method and the collision warning method described in the above embodiments.

[0252] This disclosure also provides a non-transitory computer-readable storage medium, comprising: a computer program product stored on the non-transitory computer-readable storage medium. The computer program product includes computer program instructions, which, when executed on a computer (e.g., a display device, a terminal device), cause the computer to perform the vehicle speed detection method and the collision warning method provided in the above embodiments.

[0253] The present disclosure also provides a computer program product. The computer program product comprises computer program instructions, which, when executed on a computer (e.g., a display device, a terminal device), cause the computer to perform the vehicle speed detection method provided by the above embodiments and perform the vehicle collision warning method provided by the above embodiments.

[0254] The present disclosure also provides a computer program. When the computer program is executed on a computer (e.g., a display device, a terminal device), the computer program causes the computer to perform the vehicle speed detection method provided by the above embodiments and perform the vehicle collision warning method provided by the above embodiments.

[0255] The above is merely a specific implementation of the present disclosure, but the protection scope of the present disclosure is not limited thereto, and any person skilled in the art who thinks of changes or replacements within the technical range disclosed by the present disclosure should be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A vehicle speed detection method characterized by, The method comprises the following steps: obtaining a monitoring video of a road, and extracting a plurality of continuous frames of images in the monitoring video; identifying a vehicle in the plurality of frames of images, establishing a driving track of a target vehicle, and after establishing the driving track of the target vehicle, extracting a feature vector of the target vehicle by using a re-identification model; calculating a cosine of an angle between the feature vectors of the target vehicle in each two adjacent frames of images; judging whether the value of the cosine of the angle is greater than a similarity threshold value for G times in succession; based on the value of the cosine of the angle being greater than the similarity threshold value for G times in succession, establishing an identity information list of the corresponding target vehicle in a vehicle information retrieval library, and storing the feature vector of the target vehicle into the identity information list of the corresponding target vehicle; wherein the identity information list of the target vehicle comprises identity information of the target vehicle; obtaining image position coordinates of the target vehicle in each frame of image; obtaining world position coordinates of the target vehicle in a real world according to the image position coordinates; calculating a moving distance of the target vehicle in the real world in each two adjacent frames of images in the plurality of frames of images according to the world position coordinates; calculating a speed of the target vehicle in a current frame according to a driving distance of the target vehicle in the real world and a frame rate of the monitoring video, wherein the driving distance is obtained according to the moving distance.

2. The vehicle speed detection method according to claim 1, characterized by, The method for calculating the speed of the target vehicle in the current frame according to the driving distance of the target vehicle in the real world and the frame rate of the monitoring video comprises the following steps: calculating a sum of the moving distances of the target vehicle in the real world in each two adjacent frames of images in a current frame of image and N frames of images before the current frame of image, and taking the sum of the moving distances as the driving distance; obtaining a time difference between the current frame of image and the Nth frame of image before the current frame of image according to the frame rate of the monitoring video; obtaining a calculated speed of the target vehicle in the current frame according to the driving distance and the time difference, and taking the calculated speed of the target vehicle in the current frame as the speed of the target vehicle in the current frame; or calculating a sum of the moving distances of the target vehicle in the real world in each two adjacent frames of images in a current frame of image and N frames of images before the current frame of image, and taking the sum of the moving distances as the driving distance; obtaining a time difference between the current frame of image and the Nth frame of image before the current frame of image according to the frame rate of the monitoring video; obtaining a calculated speed of the target vehicle in the current frame according to the driving distance and the time difference; performing data smoothing processing on the calculated speed of the target vehicle in the current frame and the calculated speeds of the target vehicle in M frames of images before the current frame of image, and obtaining a result of the data smoothing processing as the speed of the target vehicle in the current frame.

3. The vehicle speed detection method according to claim 1 or 2, characterized by, Before calculating the moving distance of the target vehicle in the real world in each two adjacent frames of images in the plurality of frames of images according to the world position coordinates, the method further comprises the following steps: calculating a moving distance of the target vehicle in the image in each two adjacent frames of images in the plurality of frames of images; determining whether the moving distance of the target vehicle in the image is greater than a distance threshold value; based on the moving distance of the target vehicle in the image being greater than the distance threshold value, calculating the moving distance of the target vehicle in the real world; based on the moving distance of the target vehicle in the image being less than or equal to the distance threshold value, continuing to track the driving trajectory of the target vehicle.

4. The vehicle speed detection method according to claim 3, characterized by, Before calculating the speed of the target vehicle in the current frame, further comprising: calculating the number of image moving distances greater than the distance threshold value in a plurality of image moving distances in a plurality of frames of images before the current frame of image; wherein each image moving distance is the moving distance of the target vehicle in the image between each adjacent two frames of images; determining whether the number of image moving distances greater than the distance threshold value is greater than a set threshold value; based on the number of image moving distances greater than the distance threshold value being greater than the set threshold value, calculating the speed of the target vehicle in the current frame; based on the number of image moving distances greater than the distance threshold value being less than or equal to the set threshold value, tracking the driving trajectory of the target vehicle.

5. The vehicle speed detection method according to claim 4, characterized by, The vehicle speed detection method further comprises: determining whether the speed of the target vehicle in each frame of the continuous L frames of images is outside the driving speed limit range of the road on which the target vehicle travels; based on the speed of the target vehicle in each frame of the continuous L frames of images being outside the driving speed limit range of the road on which the target vehicle travels, performing a warning operation; based on the speed of the target vehicle in at least one frame of the continuous L frames of images being within the driving speed limit range of the road on which the target vehicle travels, not performing a warning operation; after the target vehicle triggers a warning when performing speed detection in a plurality of monitoring videos, if the target vehicle triggers a warning when performing speed detection again, the warning intensity is increased.

6. The vehicle speed detection method according to claim 5, characterized by Before identifying the target vehicle in the plurality of frames of images and establishing the driving trajectory of the target vehicle, the vehicle speed detection method further comprises: labeling a detection area on the plurality of frames of images; wherein the detection area is a closed figure located in the driving area of the road in the image, and the image position coordinates of the boundary of the detection area on each frame of image are fixed; the calculation of the moving distance of the target vehicle in the real world in adjacent two frames of images in the plurality of frames of images based on the world position coordinates comprises: the calculation of the moving distance of the target vehicle in the real world in adjacent two frames of images in the plurality of frames of images located in the detection area based on the world position coordinates; the calculation of the speed of the target vehicle in the current frame comprises: the calculation of the speed of the target vehicle in the current frame located in the detection area.

7. The vehicle speed detection method according to claim 6, characterized by, The identification of the target vehicle in the plurality of frames of images and the establishment of the driving trajectory of the target vehicle comprise: detecting the plurality of frames of images to determine the target vehicle and establish a target detection box, tracking the target vehicle based on the target detection box, and obtaining the motion trajectory of the target vehicle; the image position coordinates of the target vehicle in each frame of image are the image position coordinates of the center point of the target detection box of the target vehicle in the frame of image.

8. The vehicle speed detection method according to claim 6, characterized by, In the case that the target vehicle tracking is lost, it is judged whether the target detection frame of the target vehicle is located in the detection area in the image frame before the target vehicle tracking is lost; Based on the target detection frame of the target vehicle being located in the detection area, the feature vector of the target vehicle whose tracking is lost is matched with the feature vector of the target vehicle newly acquired after the tracking is lost, and the identity information of the newly acquired target vehicle which is consistent with the feature vector matching is established in the identity information list of the target vehicle whose tracking is lost; Based on the target detection frame of the target vehicle being located outside the detection area, the target vehicle is stopped from being found back.

9. The vehicle speed detection method according to claim 7, characterized by, The detection on the multiple frames of images, the determination of the target vehicle and the establishment of the target detection frame, the tracking of the target vehicle based on the target detection frame, and the obtaining of the motion trajectory of the target vehicle further include: The target detection frame has multiple target detection frames, and in the case that two or more target detection frames overlap, the feature vector of the target vehicle corresponding to the overlapping target detection frame is extracted; The target vehicle whose feature vector has a cosine value greater than a similarity threshold in adjacent two frames of images is established in the same identity information list.

10. The vehicle speed detection method according to any one of claims 1, 2, 4, 5, 6, 7, 8, 9, characterized by, Before the world position coordinates of the target vehicle in the real world are obtained according to the image position coordinates, the method further includes: calculating internal parameters and external parameters of an image acquisition device used for shooting the monitoring video; the internal parameters and the external parameters are used for converting the image position coordinates of the multiple frames of images and the corresponding world position coordinates thereof; The calculation of the internal parameters and the external parameters of the image acquisition device used for shooting the monitoring video includes: annotating a first vanishing point and a second vanishing point on a labeled image in the multiple frames of images; wherein the labeled image is any frame in the multiple frames of images; obtaining image position coordinates of the first vanishing point and the second vanishing point in the labeled image; establishing a straight line equation passing through the first vanishing point and the second vanishing point; coinciding the center of the labeled image with a principal point, and calculating initial internal parameters and initial external parameters of the image acquisition device according to the straight line equation; selecting at least one calibration reference on the labeled image, the calibration reference being a marker with a known distance between two ends in the real world, and the calibration reference including one segment of a dashed lane line, a spacing line between adjacent dashed lane lines, or a spacing line between two connected segments of the same dashed lane line; obtaining image position coordinates of two end points of the calibration reference in the labeled image; taking the at least one calibration reference as a constraint condition to construct a constraint formula, and iteratively processing the initial internal parameters and the initial external parameters according to the constraint formula, so as to obtain the internal parameters and the external parameters of the image acquisition device according to the optimal solution of the constraint formula; The constraint formula is: wherein N is the number of the calibration references, P K is the world position coordinate of one end of the kth calibration reference in the real world, Q K is the world position coordinate of the other end of the kth calibration reference in the real world; is the image position coordinate of one end of the kth calibration reference in the annotation image, the world position coordinate of the kth calibration reference in the real world calculated by using the initial intrinsic parameters and the initial extrinsic parameters, is the image position coordinate of the other end of the kth calibration reference in the annotation image, the world position coordinate of the kth calibration reference in the real world calculated by using the initial intrinsic parameters and the initial extrinsic parameters; and cp represents the constraint parameters of the image acquisition device, including the intrinsic parameters and the extrinsic parameters.

11. A collision warning method characterized by comprising: The vehicle collision warning method includes using the vehicle speed detection method according to any one of claims 1 to 10, acquiring a monitoring video of a road, and extracting multiple continuous frames of images in the monitoring video. Target detection and tracking, and speed detection are performed on the target vehicles; The vehicle collision warning method further comprises: In the monitoring video of the road, the motion trajectories of a plurality of target vehicles are established; It is judged whether the motion trajectories of at least two adjacent target vehicles in the plurality of target vehicles are the same motion trajectory; If the motion trajectories of the two adjacent target vehicles are the same motion trajectory, the speed of each frame of the two adjacent target vehicles in the plurality of images is detected, and it is judged whether the speed of the rear target vehicle is greater than the speed of the front target vehicle in a preset duration; Based on the fact that the speed of the rear target vehicle is greater than the speed of the front target vehicle in a preset duration, a vehicle collision warning operation is performed; Based on the fact that the speed of the rear target vehicle is less than or equal to the speed of the front target vehicle in a preset duration, a vehicle collision warning operation is not performed.

12. The collision warning method of claim 11 wherein, The vehicle collision warning method further comprises: if the motion trajectories of W adjacent target vehicles are the same motion trajectory, the vehicle types of the W adjacent target vehicles are obtained, and W is greater than or equal to 3; It is judged whether at least one small or medium-sized vehicle is located between the two large vehicles; Based on the fact that at least one small or medium-sized vehicle is located between the two large vehicles, a vehicle collision warning operation is performed; Based on the fact that at least one small or medium-sized vehicle is not located between the two large vehicles, the speed of each frame of the W adjacent target vehicles in the plurality of images is detected, and it is judged whether the speed of the rear target vehicle is greater than the speed of the front target vehicle in a preset duration; Based on the fact that the speed of the rear target vehicle is greater than the speed of the front target vehicle in a preset duration, a vehicle collision warning operation is performed; Based on the fact that the speed of the rear target vehicle is less than or equal to the speed of the front target vehicle in a preset duration, a vehicle collision warning operation is not performed.

13. The collision warning method according to claim 11 or 12, characterized by, The method for judging whether the motion trajectories of at least two target vehicles are the same motion trajectory comprises: An image position coordinate set of each target vehicle in the plurality of images is obtained, and a straight line equation is fitted to obtain a motion straight line equation of each target vehicle in an image coordinate system, wherein the origin of the image coordinate system coincides with the center of each frame of image; It is judged whether the difference between the slopes of at least two motion straight line equations is less than a slope threshold value, and the difference between the intercepts of the at least two motion straight line equations is less than an intercept threshold value; Based on the fact that the difference between the slopes of at least two motion straight line equations is less than a slope threshold value, and the difference between the intercepts of the at least two motion straight line equations is less than an intercept threshold value, it is determined that the motion trajectories of the at least two target vehicles corresponding to the at least two motion straight line equations are the same motion trajectory.

14. The collision warning method of claim 13 wherein, When the same target vehicle triggers a collision warning in a plurality of collision warning in the monitoring videos, and the target vehicle triggers a collision warning again in the collision warning, the alarm intensity is increased.

15. An electronic device, comprising: The method comprises the steps of: a processor and a memory; the processor is configured to perform the following steps: obtaining monitoring videos of a road and storing the monitoring videos in the memory; extracting a plurality of continuous frames of images in the monitoring videos; identifying a target vehicle in the plurality of frames of images and establishing a driving track of the target vehicle; after establishing the driving track of the target vehicle, a re-identification model is used to extract a feature vector of the target vehicle; calculating a cosine of an angle between the feature vectors of the target vehicle in each two adjacent frames of images; judging whether the value of the cosine of the angle is greater than a similarity threshold value for G times continuously; based on the value of the cosine of the angle being greater than the similarity threshold value for G times continuously, establishing an identity information list of the corresponding target vehicle in a vehicle information retrieval library and storing the feature vector of the target vehicle in the identity information list of the corresponding target vehicle; wherein the identity information list of the target vehicle comprises identity information of the target vehicle; obtaining a world position coordinate of the target vehicle in a real world according to the image position coordinate; calculating a moving distance of the target vehicle in the real world in two adjacent frames of images of the monitoring videos of the road; calculating a speed of the target vehicle in the current frame according to a driving distance of the target vehicle in the real world and a frame rate of the monitoring videos, wherein the driving distance is obtained according to the moving distance.

16. The electronic device of claim 15, wherein, the processor is further configured to perform the following steps: calculating a sum of moving distances of the target vehicle in the real world in each two adjacent frames of images in the current frame of image and N frames of images before the current frame of image, and taking the sum of the moving distances as the driving distance; obtaining a time difference between the current frame of image and the Nth frame of image before the current frame of image according to the frame rate of the monitoring videos; obtaining a calculated speed of the target vehicle in the current frame according to the driving distance and the time difference, and taking the calculated speed of the target vehicle in the current frame as the speed of the target vehicle in the current frame; or, the processor is further configured to perform the following steps: calculating a sum of moving distances of the target vehicle in the real world in each two adjacent frames of images in the current frame of image and N frames of images before the current frame of image, and taking the sum of the moving distances as the driving distance; obtaining a time difference between the current frame of image and the Nth frame of image before the current frame of image according to the frame rate of the monitoring videos; obtaining a calculated speed of the target vehicle in the current frame according to the driving distance and the time difference; performing data smoothing processing on the calculated speed of the target vehicle in the current frame and the calculated speed of each frame of image in M frames of images before the current frame of image, and obtaining a result of the data smoothing processing as the speed of the target vehicle in the current frame.

17. The electronic device of claim 15 or 16, wherein, Before the processor is configured to calculate the moving distance of the target vehicle in the real world according to the world position coordinates, the processor is further configured to perform the following steps: calculating the moving distance of the target vehicle in the image in adjacent two frames of the monitoring video of the road; judging whether the moving distance of the target vehicle in the image is greater than a distance threshold value; calculating the moving distance of the target vehicle in the real world based on that the moving distance of the target vehicle in the image is greater than the distance threshold value; and continuing to track the driving trajectory of the target vehicle based on that the moving distance of the target vehicle in the image is less than or equal to the distance threshold value.

18. The electronic device of claim 17, wherein, Before the processor is configured to calculate the speed of the target vehicle in the current frame, the processor is further configured to perform the following steps: calculating the number of image moving distances greater than the distance threshold value in a plurality of image moving distances in a plurality of frames of images before the current frame of image; wherein each image moving distance is the moving distance of the target vehicle in the image in each adjacent two frames of image; judging whether the number of image moving distances greater than the distance threshold value is greater than a set threshold value; calculating the speed of the target vehicle in the current frame based on that the number of image moving distances greater than the distance threshold value is greater than the set threshold value; and tracking the driving trajectory of the target vehicle based on that the number of image moving distances greater than the distance threshold value is less than or equal to the set threshold value.

19. The electronic device of claim 18, wherein, The processor is further configured to perform the following steps: judging whether the speed of the target vehicle in each of the continuous L frames of image is outside the driving speed limit range of the road on which the target vehicle travels; performing a pre-warning operation based on that the speed of the target vehicle in each of the continuous L frames of image is outside the driving speed limit range of the road on which the target vehicle travels; not performing a pre-warning operation based on that the speed of the target vehicle in at least one of the continuous L frames of image is within the driving speed limit range of the road on which the target vehicle travels; increasing the warning intensity when the target vehicle triggers a pre-warning again when performing speed detection again after the target vehicle triggers a pre-warning when performing speed detection in a plurality of the monitoring videos.

20. The electronic device of claim 15 or 16, wherein, The processor is further configured to perform the following steps: obtaining the motion trajectories of a plurality of target vehicles in the monitoring video of the road; judging whether the motion trajectories of at least two adjacent target vehicles among the plurality of target vehicles are the same motion trajectory; if the motion trajectories of the adjacent two target vehicles are the same motion trajectory, detecting the speed of the target vehicle in the current frame of the plurality of frames of image of the adjacent two target vehicles, and judging whether the speed of the rear target vehicle is continuously greater than the speed of the front target vehicle within a preset duration; performing a collision pre-warning operation based on that the speed of the rear target vehicle is continuously greater than the speed of the front target vehicle within the preset duration; and not performing a collision pre-warning operation based on that there is a case that the speed of the rear target vehicle is less than or equal to the speed of the front target vehicle within the preset duration.

21. The electronic device of claim 20, wherein, If the motion trajectories of the adjacent W target vehicles are the same motion trajectory, and W is greater than or equal to 3, the processor is further configured to perform the following steps: acquiring the vehicle types of the adjacent W target vehicles, judging whether the vehicle types of the adjacent W target vehicles are at least one small or medium-sized vehicle located between two large-sized vehicles; based on the case that at least one small or medium-sized vehicle is located between the two large-sized vehicles, performing a collision warning operation; based on the case that at least one small or medium-sized vehicle is not located between the two large-sized vehicles, detecting the speed of each frame of the adjacent W target vehicles in the multiple frames of images, and judging whether the speed of the rear target vehicle is greater than the speed of the front target vehicle in the adjacent W target vehicles within a preset duration; based on the case that the speed of the rear target vehicle is greater than the speed of the front target vehicle in the adjacent W target vehicles within a preset duration, performing a collision warning operation; based on the case that the speed of the rear target vehicle is less than or equal to the speed of the front target vehicle in the adjacent W target vehicles within a preset duration, not performing a collision warning operation.

22. A vehicle warning system, characterized by The electronic device of any one of claims 15 to 21; a plurality of image acquisition devices electrically connected to the electronic device, the plurality of image acquisition devices being installed near the road, the plurality of image acquisition devices being used to shoot monitoring videos of the road and upload data of the monitoring videos to the electronic device; wherein the plurality of image acquisition devices are electrically connected to the processor in the electronic device, and the processor stores the received monitoring videos in the memory in the electronic device. The computer program product stored on a non-transitory computer readable storage medium, the computer program product comprising computer program instructions, when the computer program instructions are executed on a computer, the computer program instructions cause the computer to perform the vehicle speed detection method of any one of claims 1 to 10 and perform the collision warning method of any one of claims 11 to 14.

23. A non-transitory computer-readable storage medium, comprising: ​ ​

Images