Vehicle congestion detection method, device, electronic device and storage medium

Abstract

The present invention relates to the field of vehicle congestion detection technology, and in particular to a vehicle congestion detection method, device, electronic device and storage medium, the method comprising obtaining a current road condition image of a target detection area and the total area of ​​the target detection area, as well as a historical road condition image with a preset time interval from the current moment; inputting the current road condition image and the historical road condition image into a vehicle annotation model to annotate vehicles in the target detection area, and obtaining a current vehicle annotation image and a historical vehicle annotation image; calculating a vehicle congestion index of the target detection area according to the total area of ​​the target detection area, the vehicle identification corresponding to the historical vehicle annotation image, and the vehicle identification and vehicle area corresponding to the current vehicle annotation image. Through the method of the present application, the vehicle congestion index can be detected, avoiding the situation where the detection result is inaccurate due to the inability of the historical road condition data to accurately reflect the current actual road condition, and improving the accuracy rate.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle congestion detection, and in particular to a vehicle congestion detection method, device, electronic equipment and storage medium. Background Art

[0002] At present, vehicle congestion has become an increasingly prominent problem in major cities. The detection results of vehicle congestion directly affect the subsequent management of vehicle congestion. In traditional vehicle congestion detection methods, the periods with higher frequency of congestion in the historical road condition data of toll stations are usually determined as congestion periods.

[0003] However, in actual situations, emergencies such as traffic accidents and road maintenance can also cause vehicle congestion. Therefore, historical road condition data cannot accurately reflect the current actual road conditions, resulting in inaccurate detection results of traditional vehicle congestion detection methods and low accuracy. Summary of the invention

[0004] In view of this, the purpose of the present application is to provide a vehicle congestion detection method, device, electronic device and storage medium, which can detect the vehicle congestion index, avoid the situation where the detection results are inaccurate due to the inability of historical road condition data to accurately reflect the current actual road conditions, and improve the accuracy.

[0005] In a first aspect, an embodiment of the present application provides a method for detecting vehicle congestion, the method for detecting vehicle congestion comprising:

[0006] Obtain the current traffic condition image of the target detection area and the total area of ​​the target detection area, as well as the historical traffic condition image with a preset time interval from the current time;

[0007] The current road condition image and the historical road condition image are respectively input into the vehicle annotation model to annotate the vehicles in the target detection area, and obtain the current vehicle annotation image and the historical vehicle annotation image; the vehicle annotation model is trained based on the target road condition sample image and the corresponding vehicle annotation label image; the target road condition sample image is determined based on the brightness of each initial road condition sample image and the difference value between the initial road condition sample image and the corresponding previous frame initial road condition sample image;

[0008] The vehicle congestion index of the target detection area is calculated based on the total area of ​​the target detection area, the vehicle identification corresponding to the historical vehicle annotated image, and the vehicle identification and vehicle area corresponding to the current vehicle annotated image; the vehicle congestion index combines the vehicle retention rate and vehicle area ratio corresponding to the current vehicle annotated image.

[0009] In a possible implementation, the vehicle labeling model includes a convolutional layer, a first intermediate layer, a second intermediate layer, a third intermediate layer, a fourth intermediate layer, and an output layer; the current road condition image or the historical road condition image is input into the vehicle labeling model to label the vehicles in the target detection area, and the current vehicle labeling image or the historical vehicle labeling image is obtained, including:

[0010] Inputting the current road condition image or the historical road condition image into the convolution layer and the first intermediate layer in sequence to obtain the first image feature;

[0011] Inputting the first image feature into the second intermediate layer to obtain a first target local image feature in the first image feature;

[0012] Inputting the first target local image feature into the third intermediate layer to obtain the second target local image feature in the first target local image feature;

[0013] The second target local image features are sequentially input into the fourth intermediate layer and the output layer to obtain a current vehicle annotated image or a historical vehicle annotated image.

[0014] In a possible implementation, the first image feature is input into the second intermediate layer to obtain a first target local image feature in the first image feature, including:

[0015] Dividing the first image feature according to the size of a first preset sliding window to obtain a plurality of first initial local image features;

[0016] Calculate the attention weight corresponding to each first initial local image feature according to the attention values ​​corresponding to all first initial local image features and the current mask vector;

[0017] Determine the first initial local image feature with the largest attention weight as the first target local image feature;

[0018] If the current number of the first target local image features is less than the first preset number, the mask vector corresponding to each first initial local image feature is updated according to the attention weight corresponding to each first initial local image feature; and jump to the step of calculating the attention weight corresponding to each first initial local image feature according to the attention value corresponding to all first initial local image features and the current mask vector to continue execution.

[0019] In a possible implementation manner, the target road condition sample image is determined by the following steps:

[0020] Obtaining an initial road condition sample image corresponding to each frame in the road condition sample video;

[0021] Determine the first frame of initial road condition sample image as the target road condition sample image;

[0022] For each initial traffic condition sample image except the first frame of the initial traffic condition sample image, calculate the brightness of the initial traffic condition sample image and the difference between the previous frame of the initial traffic condition sample image and the initial traffic condition sample image;

[0023] Determine the key index of the initial road condition sample image according to the brightness of the initial road condition sample image and the difference value between the initial road condition sample image and the initial road condition sample image of the previous frame corresponding to the initial road condition sample image;

[0024] If the key index is greater than or equal to the preset key index, the initial road condition sample image is determined as the target road condition sample image.

[0025] In a possible implementation, the brightness of the initial road condition sample image is calculated using the following formula:

[0026] ;

[0027] in, is the i-th initial road condition sample image Brightness, is the initial road condition sample image The horizontal coordinate of the pixel point in is the initial road condition sample image The vertical coordinate of the pixel point in the is the i-th initial road condition sample image The median coordinate is The pixel value of the pixel point, is the i-th initial road condition sample image Height, is the i-th initial road condition sample image Width.

[0028] In a possible implementation, the vehicle congestion index of the target detection area is calculated according to the total area of ​​the target detection area, the vehicle identification corresponding to the historical vehicle annotated image, and the vehicle identification and vehicle area corresponding to the current vehicle annotated image, including:

[0029] Calculate the vehicle retention rate corresponding to the current vehicle annotated image based on the number of vehicle identifications contained in the vehicle identifications corresponding to the historical vehicle annotated images and the vehicle identifications corresponding to the current vehicle annotated image, and the total number of vehicle identifications after duplicate removal;

[0030] Calculate the ratio of the vehicle area corresponding to the current vehicle annotated image to the total area of ​​the target detection area to obtain the vehicle area ratio corresponding to the current vehicle annotated image;

[0031] The vehicle retention rate and vehicle area ratio corresponding to the current vehicle annotated image are weighted and summed to obtain the initial vehicle congestion index of the target detection area;

[0032] According to the initial vehicle congestion index of the target detection area, a target vehicle congestion index of the target detection area is calculated.

[0033] In a possible implementation, calculating a target vehicle congestion index of the target detection area according to an initial vehicle congestion index of the target detection area includes:

[0034] Substitute the initial vehicle congestion index of the target detection area into the following formula to obtain the target vehicle congestion index of the target detection area:

[0035] ;

[0036] in, is the target vehicle congestion index in the target detection area, is the vehicle retention rate corresponding to the current vehicle annotated image, is the vehicle area ratio corresponding to the current vehicle annotation image, is the initial vehicle congestion index obtained by weighted summing of the vehicle retention rate and vehicle area ratio corresponding to the current vehicle annotated image. is the preset weight, The preset weights.

[0037] In a second aspect, an embodiment of the present application further provides a vehicle congestion detection device, the vehicle congestion detection device comprising:

[0038] An acquisition module is used to acquire a current road condition image of a target detection area and a total area of ​​the target detection area, as well as a historical road condition image with a preset time interval from the current moment;

[0039] The input module is used to input the current road condition image and the historical road condition image into the vehicle annotation model to annotate the vehicles in the target detection area, thereby obtaining the current vehicle annotation image and the historical vehicle annotation image; the vehicle annotation model is trained based on the target road condition sample image and the corresponding vehicle annotation label image; the target road condition sample image is determined based on the brightness of each initial road condition sample image and the difference between the initial road condition sample image and the corresponding previous frame initial road condition sample image;

[0040] The calculation module is used to calculate the vehicle congestion index of the target detection area according to the total area of ​​the target detection area, the vehicle identification corresponding to the historical vehicle annotated image, and the vehicle identification and vehicle area corresponding to the current vehicle annotated image; the vehicle congestion index combines the vehicle retention rate and vehicle area ratio corresponding to the current vehicle annotated image.

[0041] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the storage medium communicate via the bus, and the processor executes the machine-readable instructions to perform the steps of any vehicle congestion detection method as in the first aspect.

[0042] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the vehicle congestion detection method as described in any one of the first aspects are executed.

[0043] The embodiment of the present application provides a method for detecting vehicle congestion, which includes obtaining a current road condition image of a target detection area and a total area of ​​the target detection area, as well as a historical road condition image with a preset time interval from the current moment; respectively inputting the current road condition image and the historical road condition image into a vehicle annotation model to annotate the vehicles in the target detection area, and obtaining a current vehicle annotation image and a historical vehicle annotation image; the vehicle annotation model is trained based on the target road condition sample image and the corresponding vehicle annotation label image; the target road condition sample image is determined based on the brightness of each initial road condition sample image and the difference between the initial road condition sample image and the corresponding previous frame initial road condition sample image; according to the total area of ​​the target detection area, the vehicle identification corresponding to the historical vehicle annotation image, and the vehicle identification and vehicle area corresponding to the current vehicle annotation image, the vehicle congestion index of the target detection area is calculated; the vehicle congestion index combines the vehicle retention rate and vehicle area ratio corresponding to the current vehicle annotation image. Through the method of the present application, the vehicle congestion index can be detected, avoiding the situation where the detection result is inaccurate due to the inability of the historical road condition data to accurately reflect the current actual road conditions, thereby improving the accuracy rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 A flow chart of a method for detecting vehicle congestion provided in an embodiment of the present application is shown;

[0046] Figure 2 A network structure diagram of a vehicle annotation model provided in an embodiment of the present application is shown;

[0047] Figure 3A schematic diagram of the structure of a vehicle congestion detection device provided in an embodiment of the present application is shown;

[0048] Figure 4 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0049] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0050] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0051] In order to enable those skilled in the art to use the contents of this application, the following implementation is provided in conjunction with a specific application scenario, "vehicle congestion detection field". For those skilled in the art, the general principles defined herein may be applied to other embodiments and application scenarios without departing from the spirit and scope of this application. Although this application is mainly described around the "vehicle congestion detection field", it should be understood that this is only an exemplary embodiment.

[0052] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0053] A method for detecting vehicle congestion provided in an embodiment of the present application is described in detail below.

[0054] Reference Figure 1As shown, it is a flow chart of a method for detecting vehicle congestion provided in an embodiment of the present application. The specific execution process of the method for detecting vehicle congestion is as follows:

[0055] S101, obtaining a current road condition image of a target detection area and a total area of ​​the target detection area, as well as a historical road condition image with a preset time interval from the current time.

[0056] S102 , respectively inputting the current road condition image and the historical road condition image into a vehicle labeling model to label vehicles in the target detection area, thereby obtaining a current vehicle labeling image and a historical vehicle labeling image.

[0057] S103, calculating a vehicle congestion index of the target detection area according to the total area of ​​the target detection area, the vehicle identification corresponding to the historical vehicle annotated image, and the vehicle identification and vehicle area corresponding to the current vehicle annotated image.

[0058] The embodiment of the present application provides a method for detecting vehicle congestion, which includes obtaining a current road condition image of a target detection area and a total area of ​​the target detection area, as well as a historical road condition image with a preset time interval from the current moment; respectively inputting the current road condition image and the historical road condition image into a vehicle annotation model to annotate the vehicles in the target detection area, and obtaining a current vehicle annotation image and a historical vehicle annotation image; the vehicle annotation model is trained based on the target road condition sample image and the corresponding vehicle annotation label image; the target road condition sample image is determined based on the brightness of each initial road condition sample image and the difference between the initial road condition sample image and the corresponding previous frame initial road condition sample image; according to the total area of ​​the target detection area, the vehicle identification corresponding to the historical vehicle annotation image, and the vehicle identification and vehicle area corresponding to the current vehicle annotation image, the vehicle congestion index of the target detection area is calculated; the vehicle congestion index combines the vehicle retention rate and vehicle area ratio corresponding to the current vehicle annotation image. Through the method of the present application, the vehicle congestion index can be detected, avoiding the situation where the detection result is inaccurate due to the inability of the historical road condition data to accurately reflect the current actual road conditions, thereby improving the accuracy rate.

[0059] The following is a description of the exemplary steps of the present application embodiment:

[0060] S101, obtaining a current road condition image of a target detection area and a total area of ​​the target detection area, as well as a historical road condition image with a preset time interval from the current time.

[0061] In the implementation manner of the present application, a shooting device is installed in advance in the target detection area (such as a toll booth, traffic light, etc.) where congestion detection is required, and the shooting device information is connected, so as to obtain the image captured by the shooting device in real time, and obtain the current road condition image of the target detection area; wherein, the current road condition image refers to the image corresponding to all areas that can be captured by the shooting device, and the current road condition image contains the target detection area where congestion detection is required. The total area of ​​the target detection area refers to the total area of ​​the target detection area in the image captured by the shooting device (referring to any one of the current road condition images currently captured and the historical road condition images captured by the shooting device). The historical road condition image is taken before the current road condition image captured at the current moment, and the interval between the shooting time corresponding to the historical road condition image and the current moment is a preset time.

[0062] For example, the current time is 3:30:40 and the preset duration is 5 seconds. The current road condition image is the image of the target detection area taken by the shooting device at 3:30:40, and the historical road condition image is the image of the target detection area taken by the shooting device at 3:30:35.

[0063] S102 , respectively inputting the current road condition image and the historical road condition image into a vehicle labeling model to label vehicles in the target detection area, thereby obtaining a current vehicle labeling image and a historical vehicle labeling image.

[0064] In an embodiment of the present application, the vehicle labeling model is trained based on the target road condition sample image and the corresponding vehicle label image; the target road condition sample image is determined based on the brightness of each initial road condition sample image and the difference between the initial road condition sample image and the corresponding previous frame initial road condition sample image.

[0065] Among them, the vehicle labeling model is a model (GSA-ResNeSt) obtained by embedding the progressive spatial area attention mechanism (PSPA) and the dynamic mask mechanism in the ResNeSt split attention mechanism.

[0066] Specifically, Figure 2 As shown in Table 1, it is a network structure diagram of the vehicle annotation model provided in the embodiment of the present application, wherein the vehicle annotation model includes a convolutional layer DeepStem, a first intermediate layer Layer1, a second intermediate layer Layer2, a third intermediate layer Layer3, a fourth intermediate layer Layer4, and an output layer Global Pooling (also called global pooling). Referring to Table 1, it is a network structure table of the vehicle annotation model provided in the embodiment of the present application:

[0067] Table 1

[0068]

[0069] Among them, Convotional Block is a convolution block. is convolution, is the convolution kernel, is the step length, is the number of output layer channels, is the maximum pooling layer, ResNeSt Block is the split attention mechanism module, GSA Block is an attention segmentation convolution that combines the attention mechanism and segmentation convolution. For regional embedding, is the parameter matrix, is a partial self-attention mechanism, is deconvolution, It is average pooling, and Average Pooling is the average pooling module.

[0070] Here, the current road condition image or the historical road condition image is input into the vehicle labeling model to label the vehicles in the target detection area through the following steps to obtain the current vehicle labeling image or the historical vehicle labeling image:

[0071] Step 1: Input the current road condition image or the historical road condition image Image into the convolution layer and the first intermediate layer in sequence to obtain the first image feature.

[0072] In an embodiment of the present application, the size of the current road condition image or the historical road condition image is [224, 224, 3], the size of the feature output by the convolutional layer is [112, 112, 64], and the size of the first image feature is [56, 56, 256].

[0073] Step 2: Input the first image feature into the second intermediate layer to obtain the first target local image feature in the first image feature.

[0074] In the implementation manner of the present application, the size of the first target local image feature is [28, 28, 512].

[0075] The first image feature is input into the second intermediate layer through the following steps to obtain the first target local image feature in the first image feature:

[0076] a. Divide the first image feature according to the size of the first preset sliding window to obtain a plurality of first initial local image features.

[0077] In an embodiment of the present application, the size of the first initial local image feature is the same as the size of the first preset sliding window, and the first initial local image feature is a part of the first image feature.

[0078] b. Calculate the attention weight corresponding to each first initial local image feature according to the attention values ​​corresponding to all first initial local image features and the current mask vector.

[0079] In an embodiment of the present application, the attention values ​​corresponding to all first initial local image features are calculated through a progressive spatial region attention mechanism.

[0080] Specifically, the attention values ​​corresponding to all the first initial local image features and the current mask vector are substituted into the following formula to calculate the attention weights corresponding to each first initial local image feature.

[0081] ;

[0082] in, is the attention weight corresponding to the i-th first initial local image feature, is the i-th first initial local image feature The attention value, is the current mask vector corresponding to the i-th first initial local image feature, is the number of the first initial local image features, is the jth first initial local image feature The attention value, is the jth first initial local image feature The current mask vector.

[0083] c. Determine the first initial local image feature with the largest attention weight as the first target local image feature.

[0084] d. If the current number of the first target local image features is less than the first preset number, update the mask vector corresponding to each first initial local image feature according to the attention weight corresponding to each first initial local image feature; and jump to "b. Calculate the attention weight corresponding to each first initial local image feature according to the attention value corresponding to all first initial local image features and the current mask vector" to continue execution.

[0085] Specifically, the mask vector corresponding to each first initial local image feature is updated by the following formula:

[0086] ;

[0087] in, is the attention weight corresponding to the i-th first initial local image feature, is the maximum value of the attention weights corresponding to all the first initial local image features.

[0088] Here, by updating the mask vector corresponding to each first initial local image feature, the model can be forced to pay attention to the first initial local image feature that has been determined as the first target local image feature to 0, so that the model learns additional first initial local image features, thereby reducing the redundancy of the features and ensuring that the model learns more important information.

[0089] Step three: input the first target local image feature into the third intermediate layer to obtain the second target local image feature in the first target local image feature.

[0090] In the implementation manner of the present application, the size of the second target local image feature is [14, 14, 1024].

[0091] The first target local image feature is input into the third intermediate layer through the following steps to obtain the second target local image feature in the first target local image feature:

[0092] a. Divide the first target local image feature according to the size of the second preset sliding window to obtain a plurality of second initial local image features.

[0093] b. In an embodiment of the present application, the size of the second initial local image feature is the same as the size of the first preset sliding window, and the second initial local image feature is a part of the first target local image feature.

[0094] b. Calculate the attention weight corresponding to each second initial local image feature based on the attention values ​​corresponding to all second initial local image features and the current mask vector.

[0095] In an embodiment of the present application, the attention values ​​corresponding to all second initial local image features are calculated through a progressive spatial region attention mechanism.

[0096] Specifically, the attention values ​​corresponding to all the second initial local image features and the current mask vector are substituted into the following formula to calculate the attention weight corresponding to each second initial local image feature.

[0097] ;

[0098] in, is the attention weight corresponding to the i-th second initial local image feature, is the i-th second initial local image feature The attention value, is the current mask vector corresponding to the i-th second initial local image feature, is the number of the second initial local image features, is the jth second initial local image feature The attention value, is the jth second initial local image feature The current mask vector.

[0099] c. Determine the second initial local image feature with the largest attention weight as the second target local image feature.

[0100] d. If the current number of the second target local image features is less than the second preset number, update the mask vector corresponding to each second initial local image feature according to the attention weight corresponding to each second initial local image feature; and jump to "b. Calculate the attention weight corresponding to each second initial local image feature according to the attention value corresponding to all second initial local image features and the current mask vector" to continue execution.

[0101] Specifically, the mask vector corresponding to each second initial local image feature is updated by the following formula:

[0102] ;

[0103] in, is the attention weight corresponding to the i-th second initial local image feature, is the maximum value of the attention weights corresponding to all the second initial local image features.

[0104] Here, by updating the mask vector corresponding to each second initial local image feature, the model can be forced to pay attention to the second initial local image feature that has been determined as the second target local image feature to 0, so that the model learns additional second initial local image features, thereby reducing the redundancy of the features and ensuring that the model learns more important information.

[0105] Step 4: Input the second target local image features into the fourth intermediate layer and the output layer in sequence to obtain the current vehicle annotated image or the historical vehicle annotated image.

[0106] In an implementation manner of the present application, the size of the features output by the fourth intermediate layer is [7, 7, 2048], and the size of the current vehicle annotated image or the historical vehicle annotated image is [1, 1, 2048].

[0107] Furthermore, the target road condition sample image is determined by the following steps:

[0108] i. Obtain the initial road condition sample image corresponding to each frame in the road condition sample video.

[0109] ii. Determine the first frame of the initial road condition sample image as the target road condition sample image.

[0110] iii. For each initial road condition sample image, calculate the brightness of the initial road condition sample image and the difference between the initial road condition sample image and the initial road condition sample image of the previous frame corresponding to the initial road condition sample image.

[0111] For example, the initial road condition sample image A corresponding to the first frame, the initial road condition sample image B corresponding to the second frame, and the initial road condition sample image C corresponding to the third frame in the road condition sample video have been obtained. Then, for the initial road condition sample image B corresponding to the second frame, the brightness of the initial road condition sample image B and the difference between the initial road condition sample image A and the initial road condition sample image B are calculated; for the initial road condition sample image C corresponding to the third frame, the brightness of the initial road condition sample image C and the difference between the initial road condition sample image B and the initial road condition sample image C are calculated.

[0112] Specifically, the brightness of the initial road condition sample image is calculated by the following formula:

[0113] ;

[0114] in, is the i-th initial road condition sample image Brightness, is the initial road condition sample image The horizontal coordinate of the pixel point in is the initial road condition sample image The vertical coordinate of the pixel point in the is the i-th initial road condition sample image The median coordinate is The pixel value of the pixel point, is the i-th initial road condition sample image Height, is the i-th initial road condition sample image Width.

[0115] Specifically, the difference value between the initial road condition sample image of the previous frame and the initial road condition sample image corresponding to the initial road condition sample image is calculated by the following formula:

[0116] ;

[0117] in, is the i-th initial road condition sample image The corresponding previous frame initial road condition sample image With the initial road sample image The difference between is the initial road condition sample image The horizontal coordinate of the pixel point in is the initial road condition sample image The vertical coordinate of the pixel point in the is the i-th initial road condition sample image The median coordinate is The pixel value of the pixel point, is the i-1th initial road condition sample image The median coordinate is The pixel value of the pixel point, is the i-th initial road condition sample image Height, is the i-th initial road condition sample image Width.

[0118] iv. determining a key index of the initial road condition sample image according to the brightness of the initial road condition sample image and the difference between an initial road condition sample image of a previous frame corresponding to the initial road condition sample image and the initial road condition sample image.

[0119] In an embodiment of the present application, a weighted sum is performed on the brightness of the initial road condition sample image and the difference value between the previous frame initial road condition sample image corresponding to the initial road condition sample image and the initial road condition sample image to obtain a key index of the initial road condition sample image.

[0120] v. If the key index is greater than or equal to a preset key index, the initial road condition sample image is determined as the target road condition sample image.

[0121] In the implementation manner of the present application, the larger the key index of the initial road condition sample image is, the more important the initial road condition sample image is. If the key index is greater than or equal to a preset key index, the initial road condition sample image is determined as the target road condition sample image.

[0122] Furthermore, obtaining the vehicle annotated label image includes: inputting the target road condition sample image into a label generation model to annotate the vehicles in the target road condition sample image to obtain the vehicle annotated label image.

[0123] In an embodiment of the present application, the label generation model is an RPN (‌Region Proposal Network)‌ model, which is a neural network model used for target detection.

[0124] S103, calculating a vehicle congestion index of the target detection area according to the total area of ​​the target detection area, the vehicle identification corresponding to the historical vehicle annotated image, and the vehicle identification and vehicle area corresponding to the current vehicle annotated image.

[0125] In the implementation mode of the present application, the vehicle congestion index combines the vehicle retention rate and vehicle area ratio corresponding to the current vehicle annotated image. The higher the vehicle congestion index, the more crowded the target detection area. The vehicle area corresponding to the current vehicle annotated image refers to the sum of the areas occupied by the vehicles in the current vehicle annotated image.

[0126] Specifically, the vehicle congestion index of the target detection area is calculated through the following steps:

[0127] i. Calculate the vehicle retention rate corresponding to the current vehicle annotated image based on the number of vehicle identifications contained in the vehicle identifications corresponding to the historical vehicle annotated images and the vehicle identifications corresponding to the current vehicle annotated image, and the total number of vehicle identifications after duplicate removal.

[0128] In an implementation manner of the present application, the number of vehicle identifications commonly included in the vehicle identifications corresponding to historical vehicle annotated images and the vehicle identification corresponding to the current vehicle annotated image is counted; the total number of vehicle identifications after deduplication among the vehicle identifications corresponding to historical vehicle annotated images and the vehicle identifications corresponding to the current vehicle annotated image is counted; and the ratio of the number of commonly included vehicle identifications to the total number of vehicle identifications after deduplication is calculated to obtain the vehicle retention rate corresponding to the current vehicle annotated image.

[0129] For example, the vehicle identifications corresponding to the historical vehicle annotated images include vehicle identification 1, vehicle identification 2, and vehicle identification 4; the current vehicle annotated image corresponds to vehicle identification 1, vehicle identification 2, and vehicle identification 3. Therefore, the number of vehicle identifications (vehicle identification 1, vehicle identification 2) commonly included in the vehicle identifications corresponding to the historical vehicle annotated images and the vehicle identifications corresponding to the current vehicle annotated image is 2; the total number of vehicle identifications after deduplication among the vehicle identifications corresponding to the historical vehicle annotated images and the vehicle identifications corresponding to the current vehicle annotated image is 4, namely vehicle identification 1, vehicle identification 2, vehicle identification 3, and vehicle identification 4.

[0130] Here, OC-SORT is a target tracking method, which is a technology in the field of computer vision and aims to accurately identify and track multiple moving targets in video sequences. The OC-SORT network is used to track the vehicles in the historical vehicle annotated image and the current vehicle annotated image, and the vehicle identification corresponding to the historical vehicle annotated image and the vehicle identification corresponding to the current vehicle annotated image are obtained.

[0131] ii. Calculate the ratio of the vehicle area corresponding to the current vehicle annotated image to the total area of ​​the target detection area to obtain the vehicle area ratio corresponding to the current vehicle annotated image.

[0132] iii. Perform weighted summation on the vehicle retention rate and vehicle area ratio corresponding to the current vehicle annotated image to obtain the initial vehicle congestion index of the target detection area.

[0133] iv. Calculate the target vehicle congestion index of the target detection area based on the initial vehicle congestion index of the target detection area.

[0134] Specifically, the initial vehicle congestion index of the target detection area is substituted into the following formula to obtain the target vehicle congestion index of the target detection area:

[0135] ;

[0136] in, is the target vehicle congestion index in the target detection area, is the vehicle retention rate corresponding to the current vehicle annotated image, is the vehicle area ratio corresponding to the current vehicle annotation image, is the initial vehicle congestion index obtained by weighted summing of the vehicle retention rate and vehicle area ratio corresponding to the current vehicle annotated image. is the preset weight, The preset weights.

[0137] Furthermore, the method also includes: determining the current vehicle annotated image after the target detection area is marked, the vehicle retention rate corresponding to the current vehicle annotated image, the vehicle area ratio of the vehicle retention rate corresponding to the current vehicle annotated image, and the vehicle congestion index of the target detection area as vehicle congestion detection result information.

[0138] Based on the same inventive concept, the embodiment of the present application also provides a vehicle congestion detection device corresponding to the vehicle congestion detection method. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the above-mentioned vehicle congestion detection method in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0139] Reference Figure 3 FIG. 1 is a schematic diagram of a vehicle congestion detection device provided in an embodiment of the present application, and the vehicle congestion detection device includes:

[0140] The acquisition module 301 is used to acquire the current road condition image of the target detection area and the total area of ​​the target detection area, as well as the historical road condition image with a preset time interval from the current time;

[0141] The input module 302 is used to input the current road condition image and the historical road condition image into the vehicle annotation model to annotate the vehicles in the target detection area, thereby obtaining the current vehicle annotation image and the historical vehicle annotation image; the vehicle annotation model is obtained by training based on the target road condition sample image and the corresponding vehicle annotation label image; the target road condition sample image is determined based on the brightness of each initial road condition sample image and the difference between the initial road condition sample image and the corresponding previous frame initial road condition sample image;

[0142] The calculation module 303 is used to calculate the vehicle congestion index of the target detection area according to the total area of ​​the target detection area, the vehicle identification corresponding to the historical vehicle annotated image, and the vehicle identification and vehicle area corresponding to the current vehicle annotated image; the vehicle congestion index combines the vehicle retention rate and vehicle area ratio corresponding to the current vehicle annotated image.

[0143] In a possible implementation, the vehicle labeling model includes a convolution layer, a first intermediate layer, a second intermediate layer, a third intermediate layer, a fourth intermediate layer, and an output layer; an input module 302 is specifically used to input the current road condition image or the historical road condition image into the convolution layer and the first intermediate layer in sequence to obtain a first image feature; input the first image feature into the second intermediate layer to obtain a first target local image feature in the first image feature; input the first target local image feature into the third intermediate layer to obtain a second target local image feature in the first target local image feature; input the second target local image feature into the fourth intermediate layer and the output layer in sequence to obtain a current vehicle labeling image or a historical vehicle labeling image.

[0144] In a possible implementation, the input module 302 is further used for:

[0145] Dividing the first image feature according to the size of a first preset sliding window to obtain a plurality of first initial local image features;

[0146] Calculate the attention weight corresponding to each first initial local image feature according to the attention values ​​corresponding to all first initial local image features and the current mask vector;

[0147] Determine the first initial local image feature with the largest attention weight as the first target local image feature;

[0148] If the current number of the first target local image features is less than the first preset number, the mask vector corresponding to each first initial local image feature is updated according to the attention weight corresponding to each first initial local image feature; and jump to the step of calculating the attention weight corresponding to each first initial local image feature according to the attention value corresponding to all first initial local image features and the current mask vector to continue execution.

[0149] In a possible implementation, the device also includes: a determination module 304; the determination module 304 is specifically used to obtain the initial road condition sample images corresponding to each frame in the road condition sample video; determine the first frame of the initial road condition sample image as the target road condition sample image; for each initial road condition sample image except the first frame of the initial road condition sample image, calculate the brightness of the initial road condition sample image, and the difference between the previous frame of the initial road condition sample image corresponding to the initial road condition sample image and the initial road condition sample image; determine the key index of the initial road condition sample image based on the brightness of the initial road condition sample image and the difference between the previous frame of the initial road condition sample image corresponding to the initial road condition sample image and the initial road condition sample image; if the key index is greater than or equal to the preset key index, determine the initial road condition sample image as the target road condition sample image.

[0150] In a possible implementation, the determination module 304 is further configured to:

[0151] ;

[0152] in, is the i-th initial road condition sample image Brightness, is the initial road condition sample image The horizontal coordinate of the pixel point in is the initial road condition sample image The vertical coordinate of the pixel point in the is the i-th initial road condition sample image The median coordinate is The pixel value of the pixel point, is the i-th initial road condition sample image Height, is the i-th initial road condition sample image Width.

[0153] In a possible implementation, the calculation module 303 is specifically used to calculate the vehicle retention rate corresponding to the current vehicle annotated image based on the number of vehicle identifications commonly contained in the vehicle identifications corresponding to the historical vehicle annotated images and the vehicle identifications corresponding to the current vehicle annotated image, and the total number of vehicle identifications after deduplication; calculate the ratio of the vehicle area corresponding to the current vehicle annotated image to the total area of ​​the target detection area to obtain the vehicle area ratio corresponding to the current vehicle annotated image; perform weighted summation of the vehicle retention rate and the vehicle area ratio corresponding to the current vehicle annotated image to obtain an initial vehicle congestion index of the target detection area; and calculate the target vehicle congestion index of the target detection area based on the initial vehicle congestion index of the target detection area.

[0154] In a possible implementation manner, the calculation module 303 is further configured to:

[0155] Substitute the initial vehicle congestion index of the target detection area into the following formula to obtain the target vehicle congestion index of the target detection area:

[0156] ;

[0157] in, is the target vehicle congestion index in the target detection area, is the vehicle retention rate corresponding to the current vehicle annotated image, is the vehicle area ratio corresponding to the current vehicle annotation image, is the initial vehicle congestion index obtained by weighted summing of the vehicle retention rate and vehicle area ratio corresponding to the current vehicle annotated image. is the preset weight, The preset weights.

[0158] The embodiment of the present application provides a vehicle congestion detection device, which includes an acquisition module 301, which is used to acquire a current road condition image of a target detection area and a total area of ​​the target detection area, as well as a historical road condition image with a preset time interval from the current moment; an input module 302, which is used to input the current road condition image and the historical road condition image into a vehicle annotation model to annotate vehicles in the target detection area, and obtain a current vehicle annotation image and a historical vehicle annotation image; the vehicle annotation model is trained based on a target road condition sample image and a corresponding vehicle annotation label image; the target road condition sample image is determined based on the brightness of each initial road condition sample image and the difference between the initial road condition sample image and the corresponding previous frame initial road condition sample image; a calculation module 303, which is used to calculate a vehicle congestion index of the target detection area according to the total area of ​​the target detection area, the vehicle identification corresponding to the historical vehicle annotation image, and the vehicle identification and vehicle area corresponding to the current vehicle annotation image; the vehicle congestion index combines the vehicle retention rate and vehicle area ratio corresponding to the current vehicle annotation image. The device of the present application can detect the vehicle congestion index, thereby avoiding the situation where the historical road condition data cannot accurately reflect the current actual road condition and causing inaccurate detection results, thereby improving the accuracy.

[0159] like Figure 4 As shown, an electronic device 400 provided in an embodiment of the present application includes: a processor 401, a memory 402 and a bus, the memory 402 stores machine-readable instructions executable by the processor 401, when the electronic device is running, the processor 401 and the memory 402 communicate through the bus, and the processor 401 executes the machine-readable instructions to perform the steps of the vehicle congestion detection method as described above.

[0160] Specifically, the memory 402 and the processor 401 can be general-purpose memories and processors, which are not specifically limited here. When the processor 401 runs the computer program stored in the memory 402, the vehicle congestion detection method can be executed.

[0161] Corresponding to the above-mentioned vehicle congestion detection method, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned vehicle congestion detection method are executed.

[0162] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application. In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0163] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0164] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0165] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the information processing method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0166] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for detecting vehicle congestion, characterized in that: The vehicle congestion detection method comprises: Obtain the current traffic condition image of the target detection area and the total area of ​​the target detection area, as well as the historical traffic condition image with a preset time interval from the current time; The current road condition image and the historical road condition image are respectively input into a vehicle annotation model to annotate vehicles in the target detection area, thereby obtaining a current vehicle annotation image and a historical vehicle annotation image; the vehicle annotation model is obtained by training based on a target road condition sample image and a corresponding vehicle annotation label image; the target road condition sample image is determined based on the brightness of each initial road condition sample image and the difference between each initial road condition sample image and a corresponding previous frame initial road condition sample image; The vehicle congestion index of the target detection area is calculated based on the total area of ​​the target detection area, the vehicle identification corresponding to the historical vehicle annotated image, and the vehicle identification and vehicle area corresponding to the current vehicle annotated image; the vehicle congestion index combines the vehicle retention rate and vehicle area ratio corresponding to the current vehicle annotated image.

2. The method for detecting vehicle congestion according to claim 1, characterized in that: The vehicle labeling model includes a convolutional layer, a first intermediate layer, a second intermediate layer, a third intermediate layer, a fourth intermediate layer, and an output layer; the current road condition image or the historical road condition image is input into the vehicle labeling model to label the vehicles in the target detection area, and the current vehicle labeling image or the historical vehicle labeling image is obtained, including: Inputting the current road condition image or the historical road condition image into the convolution layer and the first intermediate layer in sequence to obtain a first image feature; Inputting the first image feature into the second intermediate layer to obtain a first target local image feature in the first image feature; Inputting the first target local image feature into the third intermediate layer to obtain a second target local image feature in the first target local image feature; The second target local image feature is sequentially input into the fourth intermediate layer and the output layer to obtain the current vehicle annotated image or the historical vehicle annotated image.

3. The method for detecting vehicle congestion according to claim 2, characterized in that: The step of inputting the first image feature into the second intermediate layer to obtain a first target local image feature in the first image feature comprises: Dividing the first image feature according to the size of a first preset sliding window to obtain a plurality of first initial local image features; Calculate the attention weight corresponding to each first initial local image feature according to the attention values ​​corresponding to all first initial local image features and the current mask vector; Determine the first initial local image feature with the largest attention weight as the first target local image feature; If the current number of the first target local image features is less than the first preset number, the mask vector corresponding to each first initial local image feature is updated according to the attention weight corresponding to each first initial local image feature; and the step of jumping to the step of calculating the attention weight corresponding to each first initial local image feature according to the attention value corresponding to all first initial local image features and the current mask vector is continued.

4. The method for detecting vehicle congestion according to claim 1, characterized in that: The target road condition sample image is determined by the following steps: Obtaining an initial road condition sample image corresponding to each frame in the road condition sample video; Determine the first frame of initial road condition sample image as the target road condition sample image; For each initial traffic condition sample image except the first frame of the initial traffic condition sample image, calculate the brightness of the initial traffic condition sample image and the difference between the previous frame of the initial traffic condition sample image corresponding to the initial traffic condition sample image and the initial traffic condition sample image; Determine a key index of the initial road condition sample image according to the brightness of the initial road condition sample image and a difference value between an initial road condition sample image of a previous frame corresponding to the initial road condition sample image and the initial road condition sample image; If the key index is greater than or equal to a preset key index, the initial road condition sample image is determined as a target road condition sample image.

5. The method for detecting vehicle congestion according to claim 4, characterized in that: The brightness of the initial road condition sample image is calculated by the following formula: ； in, is the i-th initial road condition sample image Brightness, is the initial road condition sample image The horizontal coordinate of the pixel point in is the initial road condition sample image The vertical coordinate of the pixel point in the is the i-th initial road condition sample image The median coordinate is The pixel value of the pixel point, is the i-th initial road condition sample image Height, is the i-th initial road condition sample image Width.

6. The method for detecting vehicle congestion according to any one of claims 1 to 5, characterized in that: The calculating the vehicle congestion index of the target detection area according to the total area of ​​the target detection area, the vehicle identification corresponding to the historical vehicle annotated image, and the vehicle identification and vehicle area corresponding to the current vehicle annotated image includes: Calculate the vehicle retention rate corresponding to the current vehicle annotated image according to the number of vehicle identifications contained in the vehicle identifications corresponding to the historical vehicle annotated images and the vehicle identifications corresponding to the current vehicle annotated image and the total number of vehicle identifications after duplicate removal; Calculating the ratio of the vehicle area corresponding to the current vehicle annotated image to the total area of ​​the target detection area to obtain the vehicle area ratio corresponding to the current vehicle annotated image; Performing a weighted summation on the vehicle retention rate and the vehicle area ratio corresponding to the current vehicle annotated image to obtain an initial vehicle congestion index of the target detection area; According to the initial vehicle congestion index of the target detection area, a target vehicle congestion index of the target detection area is calculated.

7. The method for detecting vehicle congestion according to claim 6, characterized in that: The step of calculating the target vehicle congestion index of the target detection area according to the initial vehicle congestion index of the target detection area comprises: Substitute the initial vehicle congestion index of the target detection area into the following formula to obtain the target vehicle congestion index of the target detection area: ； in, is the target vehicle congestion index of the target detection area, is the vehicle retention rate corresponding to the current vehicle annotated image, is the vehicle area ratio corresponding to the current vehicle annotated image, is an initial vehicle congestion index obtained by weighted summing the vehicle retention rate and vehicle area ratio corresponding to the current vehicle annotated image, is the preset weight, The preset weights.

8. A vehicle congestion detection device, characterized in that: The vehicle congestion detection device comprises: An acquisition module is used to acquire a current road condition image of a target detection area and a total area of ​​the target detection area, as well as a historical road condition image with a preset time interval from the current moment; An input module is used to input the current road condition image and the historical road condition image into a vehicle annotation model to annotate the vehicles in the target detection area, thereby obtaining a current vehicle annotation image and a historical vehicle annotation image; the vehicle annotation model is trained based on a target road condition sample image and a corresponding vehicle annotation label image; the target road condition sample image is determined based on the brightness of each initial road condition sample image and the difference between each initial road condition sample image and the corresponding previous frame initial road condition sample image; A calculation module is used to calculate the vehicle congestion index of the target detection area according to the total area of ​​the target detection area, the vehicle identification corresponding to the historical vehicle annotated image, and the vehicle identification and vehicle area corresponding to the current vehicle annotated image; the vehicle congestion index combines the vehicle retention rate and vehicle area ratio corresponding to the current vehicle annotated image.

9. An electronic device, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the storage medium communicate via the bus, and the processor executes the machine-readable instructions to perform the steps of the vehicle congestion detection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the vehicle congestion detection method according to any one of claims 1 to 7 are executed.

Images