Vehicle counting method and device, electronic equipment and storage medium

Abstract

The application relates to the technical field of artificial intelligence, and provides a vehicle counting method and device, electronic equipment and a storage medium, the method comprises the following steps: acquiring an original image collected, and performing first preprocessing on the original image to obtain a first image, wherein the original image contains a plurality of vehicles; performing a first morphological operation on the first image to obtain a plurality of contours in the original image; determining a target contour based on the plurality of contours, and performing distance transformation on the target contour to obtain a distance transformation image; performing second preprocessing on the distance transformation image to obtain a second image; performing a second morphological operation on the second image to calculate the number of vehicles in the original image. The application separates the adhered vehicles for multiple times through the morphological operation on the preprocessed image, and the accuracy of vehicle counting is improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a vehicle counting method, apparatus, electronic device, and storage medium. Background Technology

[0002] Cars are frequently used in transportation. Due to factors such as weather, road conditions, and driver skills, damage to cars is inevitable, especially in traffic accidents. Most reported accident photos involve multiple vehicles. Currently, computer vision, target detection, and other methods are widely used for intelligent damage assessment of damaged vehicles.

[0003] However, existing intelligent damage assessment systems cannot perform tasks such as counting and locating vehicles stuck together in accident photos. Therefore, accurately calculating the number of vehicles in accident photos during vehicle damage assessment has become an urgent problem for those skilled in the art. Summary of the Invention

[0004] In view of the above, it is necessary to propose a vehicle counting method, device, electronic device and storage medium. By performing morphological operations on the preprocessed image, the stuck vehicles are separated multiple times, thereby improving the accuracy of vehicle counting.

[0005] A first aspect of the present invention provides a vehicle counting method, the method comprising:

[0006] Acquire the raw image and perform a first preprocessing on the raw image to obtain a first image, wherein the raw image contains multiple vehicles;

[0007] Perform a first morphological operation on the first image to obtain multiple contours in the first image;

[0008] Determine the target contour based on the multiple contours;

[0009] Perform a distance transformation on the target contour to obtain a distance-transformed image;

[0010] The distance-transformed image is subjected to a second preprocessing step to obtain a second image;

[0011] A second morphological operation is performed on the second image to calculate the number of vehicles in the original image.

[0012] Optionally, performing a second morphological operation on the second image to calculate the number of vehicles in the original image includes:

[0013] The second image is subjected to a preset number of erosion operations, and the connected regions of the eroded image are marked.

[0014] Traverse all connected regions in the second image and calculate the number of vehicles in the original image.

[0015] Optionally, the first preprocessing of the original image to obtain the first image includes:

[0016] The original image is input into a pre-trained semantic segmentation model to obtain the segmentation result;

[0017] The segmentation result is binarized to obtain the first image.

[0018] Optionally, performing a first morphological operation on the first image to obtain multiple contours in the first image includes:

[0019] Perform a dilation operation on the first image to obtain a dilated image;

[0020] The first image is subjected to an erosion operation to obtain an eroded image;

[0021] The dilated image is subtracted from the eroded image to obtain multiple contours in the first image.

[0022] Optionally, performing a dilation operation on the first image to obtain a dilated image includes:

[0023] Retrieve pre-created struct elements;

[0024] The structuring element is used to sequentially compare and identify the first image to obtain the maximum pixel value in the image region covered by the structuring element. The gray value of the pixel at the center of the structuring element is then replaced with the maximum pixel value to obtain the dilated image.

[0025] Optionally, performing an erosion operation on the first image to obtain an eroded image includes:

[0026] The structuring element is used to sequentially compare and identify the first image to obtain the minimum pixel value in the image area covered by the structuring element. The gray value of the pixel at the center of the structuring element is then replaced with the minimum pixel value to obtain the eroded image.

[0027] Optionally, performing a distance transformation on the target contour to obtain a distance-transformed image includes:

[0028] Generate a contour mask image containing edge pixels from the target contour;

[0029] The edge pixels contained in the contour mask image are converted into corresponding distance values ​​to obtain a distance-transformed image.

[0030] A second aspect of the present invention provides a vehicle counting device, the device comprising:

[0031] An acquisition module is used to acquire the collected raw image and perform a first preprocessing on the raw image to obtain a first image, wherein the raw image contains multiple vehicles;

[0032] The first operation module is used to perform a first morphological operation on the first image to obtain multiple contours in the first image.

[0033] The determination module is used to determine the target contour based on the plurality of contours;

[0034] A transformation module is used to perform distance transformation on the target contour to obtain a distance-transformed image;

[0035] The preprocessing module is used to perform a second preprocessing on the distance-transformed image to obtain a second image;

[0036] The second operation module performs a second morphological operation on the second image to calculate the number of vehicles in the original image.

[0037] A third aspect of the present invention provides an electronic device comprising a processor and a memory, the processor being configured to implement the vehicle counting method by executing a computer program stored in the memory.

[0038] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, the computer program implementing the vehicle counting method when executed by a processor.

[0039] In summary, the vehicle counting method, apparatus, electronic device, and storage medium described in this invention can promote the construction of smart cities and be applied in fields such as smart buildings, smart security, smart communities, smart living, and the Internet of Things. By performing a first morphological operation on the first image, multiple contours in the first image are obtained. During the first morphological operation, the broken parts of the adhered vehicles are enlarged, while small and meaningless vehicle parts are eliminated, initially separating the adhered vehicles and accurately obtaining the contours in the first image. A distance transformation is performed on the target contours, and a second preprocessing is performed on the distance-transformed image to obtain a second image. A second morphological operation is then performed on the second image to calculate the number of vehicles in the original image. During the second morphological operation, a preset number of erosion operations are performed on the second image to eliminate boundary points, causing the image to shrink inward along the boundaries, further shrinking the vehicle boundaries, completely separating the adhered vehicles, and improving the accuracy of vehicle counting. Attached Figure Description

[0040] Figure 1This is a flowchart of the vehicle counting method provided in Embodiment 1 of the present invention.

[0041] Figure 2 This is a structural diagram of the vehicle counting device provided in Embodiment 2 of the present invention.

[0042] Figure 3 This is a schematic diagram of the structure of the electronic device provided in Embodiment 3 of the present invention. Detailed Implementation

[0043] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0045] Example 1

[0046] Figure 1 This is a flowchart of the vehicle counting method provided in Embodiment 1 of the present invention.

[0047] In this embodiment, the vehicle counting method can be applied to electronic devices. For electronic devices that need to count vehicles, the vehicle counting function provided by the method of this invention can be directly integrated into the electronic device, or it can run in the electronic device in the form of a software development kit (SDK). This embodiment of the invention can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain optimal results.

[0048] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, as well as machine learning and deep learning.

[0049] like Figure 1As shown, the vehicle counting method specifically includes the following steps. Depending on different needs, the order of the steps in this flowchart can be changed, and some steps can be omitted.

[0050] 101. Acquire the original image and perform a first preprocessing on the original image to obtain a first image, wherein the original image contains multiple vehicles.

[0051] In this embodiment, the original image is captured by a camera device, which can be a webcam, dashcam, mobile phone, or other device capable of capturing images. The original image contains multiple vehicles and also includes a background image.

[0052] In this embodiment, after obtaining the original image, it is necessary to perform a first preprocessing on the original image to facilitate subsequent statistics on the vehicles in the original image.

[0053] In an optional embodiment, the first preprocessing of the original image to obtain the first image includes:

[0054] The original image is input into a pre-trained semantic segmentation model to obtain the segmentation result;

[0055] The segmentation result is binarized to obtain the first image.

[0056] In this embodiment, semantic segmentation refers to labeling each pixel of an image with a corresponding category, classifying based on the labeled categories, combining image classification, object detection, and image segmentation to divide the image into blocks with certain semantic meanings, and identifying the semantic category of each block.

[0057] In this embodiment, a semantic segmentation model can be pre-trained. Specifically, the training process of the semantic segmentation model includes:

[0058] Acquire multiple historical images, wherein the multiple historical images include vehicle images corresponding to different accident backgrounds;

[0059] The vehicle outlines of each of the historical images are annotated to obtain the target historical image;

[0060] Multiple target historical images are input into the semantic segmentation model for training, so that the trained semantic segmentation model can segment vehicle outlines from vehicle images with different accident backgrounds.

[0061] In this embodiment, the vehicle image in the background of the accident may contain multiple vehicles stuck together, or vehicle parts and the body may not be in the same position. Therefore, the vehicle outline segmented by the semantic segmentation model may be the outline of a certain part of the vehicle, the outline of the body of a vehicle, or the outline of several vehicles stuck together.

[0062] In this embodiment, the semantic segmentation model can be either a dilated convolutional semantic segmentation model based on full convolution or a symmetric semantic segmentation model based on full convolution.

[0063] Among them, the fully convolutional dilated convolutional semantic segmentation model considers the spatial consistency relationship between pixels, uses multiple dilated convolutions with different sampling rates to extract features, and then fuses the features to capture contextual information of different sizes. It also uses a fully connected conditional random field (CRF) to improve the model's ability to capture details. The fully convolutional dilated convolutional semantic segmentation model improves the localization of target boundaries by using dilated convolutions, thereby improving the accuracy of the acquired vehicle contours.

[0064] The symmetric semantic segmentation model based on full convolution does not contain fully connected layers; it consists entirely of convolutional layers. By inputting an original image into a convolutional neural network, it undergoes multiple convolutions and pooling processes to obtain a series of feature maps. Then, a deconvolutional layer upsamples the feature map obtained from the last convolutional layer, ensuring that the upsampled feature map is the same size as the original image. This allows for prediction of each pixel value on the feature map while preserving its spatial location information in the original image. Finally, the upsampled feature map is classified pixel-by-pixel, and the softmax classification loss is calculated for each pixel. Because the fully convolutional symmetric semantic segmentation model does not contain fully connected layers, it adapts to original images of arbitrary sizes. Furthermore, the tiered structure combining results from different depth layers ensures the robustness and accuracy of the semantic segmentation model.

[0065] In this embodiment, semantic segmentation is performed on the original image. For example, the original image contains categories such as vehicles, trees, and people. Each pixel corresponding to a vehicle, tree, or person is labeled with its category, and the same category is segmented to obtain the category of each region block in the original image and its corresponding segmentation mask.

[0066] Specifically, the segmentation mask refers to the label corresponding to the category marked in the original image. The segmentation mask is a binary image (pixel value is 0 or 1). For example, the pixel value of the region block related to the vehicle is set to 1, and the pixel value of the region block unrelated to the vehicle is set to 0.

[0067] In this embodiment, after obtaining the segmentation result, it is necessary to perform binarization processing on the segmentation result. Specifically, the pixel value corresponding to the foreground image in the segmentation result is set to 1, and the pixel value corresponding to the background image is set to 0, to obtain a binarized image, namely the first image.

[0068] 102. Perform a first morphological operation on the first image to obtain multiple contours in the first image.

[0069] In this embodiment, the first image refers to the image after non-vehicle components are occluded, and the first image contains the outlines of multiple vehicle components.

[0070] In this embodiment, morphological operations mainly extract component information from the first image. The component information is usually used to express and describe the shape of the image. Morphological operations may include erosion, dilation, opening operation, closing operation, morphological gradient operation, etc.

[0071] In this embodiment, the contour refers to the contour obtained after performing a first morphological operation on the first image.

[0072] Specifically, performing a first morphological operation on the first image to obtain multiple contours in the first image includes:

[0073] Perform a dilation operation on the first image to obtain a dilated image;

[0074] The first image is subjected to an erosion operation to obtain an eroded image;

[0075] The dilated image is subtracted from the eroded image to obtain multiple contours in the first image.

[0076] In this embodiment, by subtracting the eroded image from the dilated image, the edges of the foreground image in the first image, i.e., multiple contours, can be obtained. Specifically, the operation type parameter op of the preset function cv2.morphologyEx() can be set to 'cv2.MORPH_GRADIENT', and the operation of subtracting the eroded image from the dilated image can be implemented using the preset function.

[0077] Further, the process of performing a dilation operation on the first image to obtain a dilated image includes:

[0078] Retrieve pre-created struct elements;

[0079] The structuring element is used to sequentially compare and identify the first image to obtain the maximum pixel value in the image region covered by the structuring element. The gray value of the pixel at the center of the structuring element is then replaced with the maximum pixel value to obtain the dilated image.

[0080] In this embodiment, structural elements can be pre-created. These structural elements can be elliptical or circular kernels, or any other shape.

[0081] In this embodiment, the dilation operation expands the boundary of the first image to fill the blanks in the segmented image, thereby enlarging the broken parts of the stuck vehicle. Specifically, during the dilation operation, structuring elements are sequentially compared and identified on the first image. The grayscale value of the pixel at the center of the structuring element is set to the largest pixel value among all pixels in the image area covered by the structuring element, where the largest pixel value can be 255.

[0082] Further, the step of performing an erosion operation on the first image to obtain an eroded image includes:

[0083] The structuring element is used to sequentially compare and identify the first image to obtain the minimum pixel value in the image area covered by the structuring element. The gray value of the pixel at the center of the structuring element is then replaced with the minimum pixel value to obtain the eroded image.

[0084] In this embodiment, the erosion operation can reduce the boundary of the first image, specifically by eroding the boundaries of vehicle components in the foreground image, thus eliminating small and meaningless vehicle components. Specifically, during the erosion operation, structuring elements are sequentially compared and identified on the first image. The grayscale value of the pixel at the center of the structuring element is set to the minimum pixel value among all pixels in the image area covered by the structuring element, where the minimum pixel value can be 0.

[0085] In this embodiment, by performing dilation and erosion operations on the first image, the shape and outline of the target in the first image are identified, the broken parts of the stuck vehicle are enlarged, and small and meaningless vehicle parts are eliminated. At the same time, by subtracting the eroded image from the dilated image, the stuck vehicle is initially separated, and the outline in the first image is accurately obtained.

[0086] 103. Determine the target contour based on the multiple contours.

[0087] In this embodiment, the contour obtained from the first image may be the contour of a vehicle or the contour of a vehicle component. In order to improve the accuracy and efficiency of vehicle counting in the future, the multiple contours are filtered, those that do not meet the vehicle area requirements are filtered, and the contours retained after filtering are determined as the target contours.

[0088] In an optional embodiment, determining the target contour based on the plurality of contours includes:

[0089] Calculate the area of ​​each contour;

[0090] The area of ​​each contour is compared with a preset contour area threshold.

[0091] Contours with an area greater than or equal to the preset contour area threshold are identified as target contours, and contours with an area less than the preset contour area threshold are deleted.

[0092] In this embodiment, the cv2.contourArea() function can be used to calculate the area of ​​each contour.

[0093] In this embodiment, a contour area threshold can be preset to filter out contours with a contour area smaller than the preset contour area threshold, that is, to filter out the contours of some smaller vehicle parts, thereby reducing the interference of small vehicle part contours and improving the efficiency and accuracy of subsequent vehicle contour counting.

[0094] 104. Perform distance transformation on the target contour to obtain a distance-transformed image.

[0095] In this embodiment, the usual pixel-level matching results are difficult to meet the contour matching requirements in positioning measurement applications. This embodiment improves the accuracy of vehicle counting by performing distance transformation on the target contour, which facilitates subsequent contour positioning.

[0096] In an optional embodiment, performing a distance transformation on the target contour to obtain a distance-transformed image includes:

[0097] Generate a contour mask image containing edge pixels from the target contour;

[0098] The edge pixels contained in the contour mask image are converted into corresponding distance values ​​to obtain a distance-transformed image.

[0099] In this embodiment, when performing distance transformation on the target contour, a contour mask image containing edge pixels is generated from the target contour. The target contour refers to a curve connecting all continuous points (along the boundary) with the same pixel value. The interior of the target contour is filled based on the pixel value, and the filled mask image is Gaussian blurred to obtain the contour mask image.

[0100] In this embodiment, the mask image is denoised by applying Gaussian blur, which makes the edges of the contour mask image smooth and ensures the quality of the contour mask image.

[0101] In this embodiment, the distance between each non-zero point (mask not equal to 0) in the contour mask image and the nearest zero point (mask equal to 0) is calculated using the distanceTransform() function in OpenCV, resulting in a distance-transformed image. The brighter the point on the distance-transformed image, the farther away it is from the zero point, and a bright fringe will be formed at the center of the vehicle. The bright fringe at the center of the vehicle represents a vehicle.

[0102] 105. Perform a second preprocessing on the distance-transformed image to obtain a second image.

[0103] In this embodiment, the second preprocessing refers to normalizing the distance transform image and performing an adaptive thresholding operation on the normalized distance transform result to obtain a new segmentation result, namely the second image.

[0104] In an optional embodiment, the second preprocessing of the distance-transformed image to obtain the second image includes:

[0105] The distance-transformed image is then normalized.

[0106] The normalization result is then subjected to threshold binarization to obtain the second image.

[0107] In this embodiment, after the distance transformation, the distance-transformed image is normalized. Specifically, the normalization process refers to converting multiple distances in the distance-transformed image into a range of [0,1].

[0108] In this embodiment, the normalization result is further binarized using an adaptive threshold segmentation algorithm. The adaptive threshold segmentation algorithm can be the OTSU algorithm. The OTSU algorithm is used to determine the optimal threshold. The pixel values ​​of all pixels in the image are traversed sequentially from top to bottom and from left to right. If the pixel value is greater than the optimal threshold, the pixel value is set to 255. If it is less than or equal to the optimal threshold, the pixel value is set to 0, thus obtaining the binarized image, i.e., the second image.

[0109] 106. Perform a second morphological operation on the second image to calculate the number of vehicles in the original image.

[0110] In this embodiment, the second morphological operation is to more accurately calculate the number of vehicles in the original image.

[0111] In an optional embodiment, performing a second morphological operation on the second image to calculate the number of vehicles in the original image includes:

[0112] The second image is subjected to a preset number of erosion operations, and the connected regions of the eroded image are marked.

[0113] Traverse all connected regions in the second image and calculate the number of vehicles in the original image.

[0114] In this embodiment, the number of etching operations can be preset, for example, 5 times or 10 times.

[0115] In this embodiment, connected region marking is completed by finding adjacent pixels with the same pixel value in the second image and setting a mark for each connected region.

[0116] Further, the step of traversing all connected regions in the second image and calculating the number of vehicles in the original image includes:

[0117] The second image is traversed for the first time, and a label is set for each pixel in the second image;

[0118] Record the connected region to which the label of each pixel belongs;

[0119] The second iteration traverses the second image, replacing the label of each pixel with the label of the connected region to which it belongs;

[0120] Group pixels with the same connected component label into a single connected component;

[0121] Each connected region is analyzed to obtain the analysis results;

[0122] Based on the analysis results, the number of vehicles in the original image is calculated.

[0123] In this embodiment, the connectedComponents() function can be used to perform connected component statistics and analysis on the second image.

[0124] Furthermore, the analysis of each connected region yields the following results:

[0125] Multiple parameters for each connected region are calculated using a preset function, wherein the parameters may be area, circumscribed rectangle, and centroid;

[0126] Determine whether the multiple parameters meet the vehicle counting conditions;

[0127] When each parameter satisfies the vehicle counting condition, the analysis result is determined to be the corresponding connected region of the record;

[0128] If any parameter does not meet the vehicle counting condition, the analysis result is determined to be that the corresponding connected region is not recorded.

[0129] In this embodiment, the original image carries vehicle counting conditions, including vehicle information such as the area of ​​the vehicle and the centroid of the vehicle.

[0130] In this embodiment, the preset function can be the connectedComponentsWithStats() function provided by OpenCV 4. The preset function is used to mark different connected regions in the second image and to count the position and area information of the connected regions.

[0131] In this embodiment, by performing a preset number of erosion operations on the second image, the boundary points in the second image are eliminated, causing the image to shrink inward along the boundary, further shrinking the vehicle boundary, and the stuck vehicles are completely separated. Furthermore, by traversing all connected regions in the second image and analyzing the connected regions, connected regions that do not meet the vehicle counting conditions are filtered out during the analysis process, thereby improving the accuracy of vehicle counting.

[0132] In summary, the vehicle counting method described in this embodiment obtains multiple contours in the first image by performing a first morphological operation on the first image. This first morphological operation expands the fragmented parts of the adhered vehicles while eliminating small and meaningless vehicle components, initially separating the adhered vehicles and accurately obtaining the contours in the first image. A distance transformation is then performed on the target contours, and a second preprocessing is applied to the distance-transformed image to obtain a second image. A second morphological operation is then performed on the second image to calculate the number of vehicles in the original image. During the second morphological operation, a preset number of erosion operations are performed on the second image to eliminate boundary points, causing the image to shrink inward along the boundaries, further shrinking the vehicle boundaries. This completely separates the adhered vehicles, improving the accuracy of vehicle counting.

[0133] Example 2

[0134] Figure 2 This is a structural diagram of the vehicle counting device provided in Embodiment 2 of the present invention.

[0135] In some embodiments, the vehicle counting device 20 may include a plurality of functional modules composed of program code segments. The program code of each program segment in the vehicle counting device 20 may be stored in the memory of the electronic device and executed by the at least one processor to perform (see details). Figure 1 (Description) The vehicle counting function.

[0136] In this embodiment, the vehicle counting device 20 can be divided into multiple functional modules according to its functions. These functional modules may include: an acquisition module 201, a first operation module 202, a determination module 203, a transformation module 204, a preprocessing module 205, and a second operation module 206. The term "module" in this invention refers to a series of computer-readable instruction segments that can be executed by at least one processor and perform a fixed function, stored in memory. In this embodiment, the functions of each module will be detailed in subsequent embodiments.

[0137] The acquisition module 201 acquires the collected original image and performs a first preprocessing on the original image to obtain a first image, wherein the original image contains multiple vehicles.

[0138] In this embodiment, the original image is captured by a camera device, which can be a webcam, dashcam, mobile phone, or other device capable of capturing images. The original image contains multiple vehicles and also includes a background image.

[0139] In this embodiment, after obtaining the original image, it is necessary to perform a first preprocessing on the original image to facilitate subsequent statistics on the vehicles in the original image.

[0140] In an optional embodiment, the acquisition module 201 performs a first preprocessing on the original image to obtain a first image, including:

[0141] The original image is input into a pre-trained semantic segmentation model to obtain the segmentation result;

[0142] The segmentation result is binarized to obtain the first image.

[0143] In this embodiment, semantic segmentation refers to labeling each pixel of an image with a corresponding category, classifying based on the labeled categories, combining image classification, object detection, and image segmentation to divide the image into blocks with certain semantic meanings, and identifying the semantic category of each block.

[0144] In this embodiment, a semantic segmentation model can be pre-trained. Specifically, the training process of the semantic segmentation model includes:

[0145] Acquire multiple historical images, wherein the multiple historical images include vehicle images corresponding to different accident backgrounds;

[0146] The vehicle outlines of each of the historical images are annotated to obtain the target historical image;

[0147] Multiple target historical images are input into the semantic segmentation model for training, so that the trained semantic segmentation model can segment vehicle outlines from vehicle images with different accident backgrounds.

[0148] In this embodiment, the vehicle image in the background of the accident may contain multiple vehicles stuck together, or vehicle parts and the body may not be in the same position. Therefore, the vehicle outline segmented by the semantic segmentation model may be the outline of a certain part of the vehicle, the outline of the body of a vehicle, or the outline of several vehicles stuck together.

[0149] In this embodiment, the semantic segmentation model can be either a dilated convolutional semantic segmentation model based on full convolution or a symmetric semantic segmentation model based on full convolution.

[0150] Among them, the fully convolutional dilated convolutional semantic segmentation model considers the spatial consistency relationship between pixels, uses multiple dilated convolutions with different sampling rates to extract features, and then fuses the features to capture contextual information of different sizes. It also uses a fully connected conditional random field (CRF) to improve the model's ability to capture details. The fully convolutional dilated convolutional semantic segmentation model improves the localization of target boundaries by using dilated convolutions, thereby improving the accuracy of the acquired vehicle contours.

[0151] The symmetric semantic segmentation model based on full convolution does not contain fully connected layers; it consists entirely of convolutional layers. By inputting an original image into a convolutional neural network, it undergoes multiple convolutions and pooling processes to obtain a series of feature maps. Then, a deconvolutional layer upsamples the feature map obtained from the last convolutional layer, ensuring that the upsampled feature map is the same size as the original image. This allows for prediction of each pixel value on the feature map while preserving its spatial location information in the original image. Finally, the upsampled feature map is classified pixel-by-pixel, and the softmax classification loss is calculated for each pixel. Because the fully convolutional symmetric semantic segmentation model does not contain fully connected layers, it adapts to original images of arbitrary sizes. Furthermore, the tiered structure combining results from different depth layers ensures the robustness and accuracy of the semantic segmentation model.

[0152] In this embodiment, semantic segmentation is performed on the original image. For example, the original image contains categories such as vehicles, trees, and people. Each pixel corresponding to a vehicle, tree, or person is labeled with its category, and the same category is segmented to obtain the category of each region block in the original image and its corresponding segmentation mask.

[0153] Specifically, the segmentation mask refers to the label corresponding to the category marked in the original image. The segmentation mask is a binary image (pixel value is 0 or 1). For example, the pixel value of the region block related to the vehicle is set to 1, and the pixel value of the region block unrelated to the vehicle is set to 0.

[0154] In this embodiment, after obtaining the segmentation result, it is necessary to perform binarization processing on the segmentation result. Specifically, the pixel value corresponding to the foreground image in the segmentation result is set to 1, and the pixel value corresponding to the background image is set to 0, to obtain a binarized image, namely the first image.

[0155] The first operation module 201 is used to perform a first morphological operation on the first image to obtain multiple contours in the first image.

[0156] In this embodiment, the first image refers to the image after non-vehicle components are occluded, and the first image contains the outlines of multiple vehicle components.

[0157] In this embodiment, morphological operations mainly extract component information from the first image. The component information is usually used to express and describe the shape of the image. Morphological operations may include erosion, dilation, opening operation, closing operation, morphological gradient operation, etc.

[0158] In this embodiment, the contour refers to the contour obtained after performing a first morphological operation on the first image.

[0159] Specifically, the first operation module 201 performs a first morphological operation on the first image to obtain multiple contours in the first image, including:

[0160] Perform a dilation operation on the first image to obtain a dilated image;

[0161] The first image is subjected to an erosion operation to obtain an eroded image;

[0162] The dilated image is subtracted from the eroded image to obtain multiple contours in the first image.

[0163] In this embodiment, by subtracting the eroded image from the dilated image, the edges of the foreground image in the first image, i.e., multiple contours, can be obtained. Specifically, the operation type parameter op of the preset function cv2.morphologyEx() can be set to 'cv2.MORPH_GRADIENT', and the operation of subtracting the eroded image from the dilated image can be implemented using the preset function.

[0164] Further, the process of performing a dilation operation on the first image to obtain a dilated image includes:

[0165] Retrieve pre-created struct elements;

[0166] The structuring element is used to sequentially compare and identify the first image to obtain the maximum pixel value in the image region covered by the structuring element. The gray value of the pixel at the center of the structuring element is then replaced with the maximum pixel value to obtain the dilated image.

[0167] In this embodiment, structural elements can be pre-created. These structural elements can be elliptical or circular kernels, or any other shape.

[0168] In this embodiment, the dilation operation expands the boundary of the first image to fill the blanks in the segmented image, thereby enlarging the broken parts of the stuck vehicle. Specifically, during the dilation operation, structuring elements are sequentially compared and identified on the first image. The grayscale value of the pixel at the center of the structuring element is set to the largest pixel value among all pixels in the image area covered by the structuring element, where the largest pixel value can be 255.

[0169] Further, the step of performing an erosion operation on the first image to obtain an eroded image includes:

[0170] The structuring element is used to sequentially compare and identify the first image to obtain the minimum pixel value in the image area covered by the structuring element. The gray value of the pixel at the center of the structuring element is then replaced with the minimum pixel value to obtain the eroded image.

[0171] In this embodiment, the erosion operation can reduce the boundary of the first image, specifically by eroding the boundaries of vehicle components in the foreground image, thus eliminating small and meaningless vehicle components. Specifically, during the erosion operation, structuring elements are sequentially compared and identified on the first image. The grayscale value of the pixel at the center of the structuring element is set to the minimum pixel value among all pixels in the image area covered by the structuring element, where the minimum pixel value can be 0.

[0172] In this embodiment, by performing dilation and erosion operations on the first image, the shape and outline of the target in the first image are identified, the broken parts of the stuck vehicle are enlarged, and small and meaningless vehicle parts are eliminated. At the same time, by subtracting the eroded image from the dilated image, the stuck vehicle is initially separated, and the outline in the first image is accurately obtained.

[0173] The determination module 203 is used to determine the target contour based on the plurality of contours.

[0174] In this embodiment, the contour obtained from the first image may be the contour of a vehicle or the contour of a vehicle component. In order to improve the accuracy and efficiency of vehicle counting in the future, the multiple contours are filtered, those that do not meet the vehicle area requirements are filtered, and the contours retained after filtering are determined as the target contours.

[0175] In an optional embodiment, the determining module 203 determines the target contour based on the plurality of contours, including:

[0176] Calculate the area of ​​each contour;

[0177] The area of ​​each contour is compared with a preset contour area threshold.

[0178] Contours with an area greater than or equal to the preset contour area threshold are identified as target contours, and contours with an area less than the preset contour area threshold are deleted.

[0179] In this embodiment, the cv2.contourArea() function can be used to calculate the area of ​​each contour.

[0180] In this embodiment, a contour area threshold can be preset to filter out contours with a contour area smaller than the preset contour area threshold, that is, to filter out the contours of some smaller vehicle parts, thereby reducing the interference of small vehicle part contours and improving the efficiency and accuracy of subsequent vehicle contour counting.

[0181] The transformation module 204 is used to perform distance transformation on the target contour to obtain a distance-transformed image.

[0182] In this embodiment, the usual pixel-level matching results are difficult to meet the contour matching requirements in positioning measurement applications. This embodiment improves the accuracy of vehicle counting by performing distance transformation on the target contour, which facilitates subsequent contour positioning.

[0183] In an optional embodiment, the transformation module 204 performs a distance transformation on the target contour to obtain a distance-transformed image, including:

[0184] Generate a contour mask image containing edge pixels from the target contour;

[0185] The edge pixels contained in the contour mask image are converted into corresponding distance values ​​to obtain a distance-transformed image.

[0186] In this embodiment, when performing distance transformation on the target contour, a contour mask image containing edge pixels is generated from the target contour. The target contour refers to a curve connecting all continuous points (along the boundary) with the same pixel value. The interior of the target contour is filled based on the pixel value, and the filled mask image is Gaussian blurred to obtain the contour mask image.

[0187] In this embodiment, the mask image is denoised by applying Gaussian blur, which makes the edges of the contour mask image smooth and ensures the quality of the contour mask image.

[0188] In this embodiment, the distance between each non-zero point (mask not equal to 0) in the contour mask image and the nearest zero point (mask equal to 0) is calculated using the distanceTransform() function in OpenCV, resulting in a distance-transformed image. The brighter the point on the distance-transformed image, the farther away it is from the zero point, and a bright fringe will be formed at the center of the vehicle. The bright fringe at the center of the vehicle represents a vehicle.

[0189] The preprocessing module 205 is used to perform a second preprocessing on the distance-transformed image to obtain a second image.

[0190] In this embodiment, the second preprocessing refers to normalizing the distance transform image and performing an adaptive thresholding operation on the normalized distance transform result to obtain a new segmentation result, namely the second image.

[0191] In an optional embodiment, the preprocessing module 205 performs a second preprocessing on the distance-transformed image to obtain a second image, including:

[0192] The distance-transformed image is then normalized.

[0193] The normalization result is then subjected to threshold binarization to obtain the second image.

[0194] In this embodiment, after the distance transformation, the distance-transformed image is normalized. Specifically, the normalization process refers to converting multiple distances in the distance-transformed image into a range of [0,1].

[0195] In this embodiment, the normalization result is further binarized using an adaptive threshold segmentation algorithm. The adaptive threshold segmentation algorithm can be the OTSU algorithm. The OTSU algorithm is used to determine the optimal threshold. The pixel values ​​of all pixels in the image are traversed sequentially from top to bottom and from left to right. If the pixel value is greater than the optimal threshold, the pixel value is set to 255. If it is less than or equal to the optimal threshold, the pixel value is set to 0, thus obtaining the binarized image, i.e., the second image.

[0196] The second operation module 206 is used to perform a second morphological operation on the second image and calculate the number of vehicles in the original image.

[0197] In this embodiment, the second morphological operation is to more accurately calculate the number of vehicles in the original image.

[0198] In an optional embodiment, the second operation module 206 performs a second morphological operation on the second image, calculating the number of vehicles in the original image, including:

[0199] The second image is subjected to a preset number of erosion operations, and the connected regions of the eroded image are marked.

[0200] Traverse all connected regions in the second image and calculate the number of vehicles in the original image.

[0201] In this embodiment, the number of etching operations can be preset, for example, 5 times or 10 times.

[0202] In this embodiment, connected region marking is completed by finding adjacent pixels with the same pixel value in the second image and setting a mark for each connected region.

[0203] Further, the step of traversing all connected regions in the second image and calculating the number of vehicles in the original image includes:

[0204] The second image is traversed for the first time, and a label is set for each pixel in the second image;

[0205] Record the connected region to which the label of each pixel belongs;

[0206] The second iteration traverses the second image, replacing the label of each pixel with the label of the connected region to which it belongs;

[0207] Group pixels with the same connected component label into a single connected component;

[0208] Each connected region is analyzed to obtain the analysis results;

[0209] Based on the analysis results, the number of vehicles in the original image is calculated.

[0210] In this embodiment, the connected components() function can be used to perform connected component statistics and analysis on the second image.

[0211] Furthermore, the analysis of each connected region yields the following results:

[0212] Multiple parameters for each connected region are calculated using a preset function, wherein the parameters may be area, circumscribed rectangle, and centroid;

[0213] Determine whether the multiple parameters meet the vehicle counting conditions;

[0214] When each parameter satisfies the vehicle counting condition, the analysis result is determined to be the corresponding connected region of the record;

[0215] If any parameter does not meet the vehicle counting condition, the analysis result is determined to be that the corresponding connected region is not recorded.

[0216] In this embodiment, the original image carries vehicle counting conditions, including vehicle information such as the area of ​​the vehicle and the centroid of the vehicle.

[0217] In this embodiment, the preset function can be the connectedComponentsWithStats() function provided by OpenCV 4. The preset function is used to mark different connected regions in the second image and to count the position and area information of the connected regions.

[0218] In this embodiment, by performing a preset number of erosion operations on the second image, the boundary points in the second image are eliminated, causing the image to shrink inward along the boundary, further shrinking the vehicle boundary, and the stuck vehicles are completely separated. Furthermore, by traversing all connected regions in the second image and analyzing the connected regions, connected regions that do not meet the vehicle counting conditions are filtered out during the analysis process, thereby improving the accuracy of vehicle counting.

[0219] In summary, the vehicle counting device described in this embodiment obtains multiple contours in the first image by performing a first morphological operation on the first image. During the first morphological operation, the broken parts of the adhered vehicles are enlarged, while small and meaningless vehicle parts are eliminated, initially separating the adhered vehicles and accurately obtaining the contours in the first image. A distance transformation is performed on the target contours, and the distance-transformed image undergoes a second preprocessing to obtain a second image. A second morphological operation is then performed on the second image to calculate the number of vehicles in the original image. During the second morphological operation, a preset number of erosion operations are performed on the second image to eliminate boundary points, causing the image to shrink inward along the boundaries, further shrinking the vehicle boundaries, completely separating the adhered vehicles, and improving the accuracy of vehicle counting.

[0220] Example 3

[0221] See Figure 3The diagram shown is a structural schematic of an electronic device provided in Embodiment 3 of the present invention. In a preferred embodiment of the present invention, the electronic device 3 includes a memory 31, at least one processor 32, at least one communication bus 33, and a transceiver 34.

[0222] Those skilled in the art should understand that Figure 3 The structure of the electronic device shown does not constitute a limitation of the embodiments of the present invention. It can be a bus structure or a star structure. The electronic device 3 may also include more or fewer other hardware or software than shown, or different component arrangements.

[0223] In some embodiments, the electronic device 3 is an electronic device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), digital processors, and embedded devices. The electronic device 3 may also include client devices, including, but not limited to, any electronic product capable of human-computer interaction with a client via a keyboard, mouse, remote control, touchpad, or voice control device, such as personal computers, tablet computers, smartphones, and digital cameras.

[0224] It should be noted that the electronic device 3 is merely an example. Other existing or future electronic products that are suitable for this invention should also be included within the scope of protection of this invention and are incorporated herein by reference.

[0225] In some embodiments, the memory 31 is used to store program code and various data, such as the vehicle counting device 20 installed in the electronic device 3, and to achieve high-speed and automatic access to programs or data during the operation of the electronic device 3. The memory 31 includes read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0226] In some embodiments, the at least one processor 32 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits packaged with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The at least one processor 32 is the control unit of the electronic device 3, connecting various components of the entire electronic device 3 via various interfaces and lines. It executes programs or modules stored in the memory 31 and calls data stored in the memory 31 to perform various functions and process data of the electronic device 3.

[0227] In some embodiments, the at least one communication bus 33 is configured to enable communication between the memory 31 and the at least one processor 32, etc.

[0228] Although not shown, the electronic device 3 may also include a power supply (such as a battery) to power the various components. Optionally, the power supply may be logically connected to the at least one processor 32 via a power management device, thereby enabling functions such as charging, discharging, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device 3 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0229] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0230] The integrated unit implemented as a software functional module described above can be stored in a computer-readable storage medium. This software functional module, stored in a storage medium, includes several instructions to cause a computer device (which may be a personal computer, electronic device, or network device, etc.) or processor to execute portions of the methods described in the various embodiments of the present invention.

[0231] In a further embodiment, combined with Figure 2 The at least one processor 32 can execute the operating device of the electronic device 3 and various installed applications (such as the vehicle counting device 20), program code, etc., for example, the various modules mentioned above.

[0232] The memory 31 stores program code, and the at least one processor 32 can call the program code stored in the memory 31 to execute related functions. For example, Figure 2 The modules described herein are program codes stored in the memory 31 and executed by the at least one processor 32, thereby realizing the functions of the modules to achieve the purpose of vehicle counting.

[0233] For example, the program code can be divided into one or more modules / units, which are stored in the memory 31 and executed by the processor 32 to complete this application. The one or more modules / units can be a series of computer-readable instruction segments capable of performing a specific function, which describe the execution process of the program code in the electronic device 3. For example, the program code can be divided into an acquisition module 201, a first operation module 202, a determination module 203, a transformation module 204, a preprocessing module 205, and a second operation module 206.

[0234] In one embodiment of the present invention, the memory 31 stores a plurality of computer-readable instructions, which are executed by the at least one processor 32 to perform a vehicle counting function.

[0235] Specifically, the specific implementation method of the above instructions by the at least one processor 32 can be referred to Figure 1 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0236] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0237] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0238] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0239] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims. Furthermore, it is clear that the word "comprising" does not exclude other elements, and the singular does not exclude the plural. Multiple elements or devices recited in the present invention may also be implemented by a single element or device in software or hardware. The terms "first," "second," etc., are used to denote names and do not indicate any particular order.

Claims

1. A vehicle counting method, characterized in that, The method includes: Acquire the raw image and perform a first preprocessing on the raw image to obtain a first image, wherein the raw image contains multiple vehicles; Perform a first morphological operation on the first image to obtain multiple contours in the first image, including: performing a dilation operation on the first image to obtain a dilated image; performing an erosion operation on the first image to obtain an eroded image; and subtracting the eroded image from the dilated image to obtain multiple contours in the first image. A target contour is determined based on the multiple contours, wherein the target contour is a curve connecting all consecutive points with the same pixel value; Performing a distance transformation on the target contour to obtain a distance-transformed image includes: filling the interior of the target contour based on the pixel values ​​of the target contour; applying Gaussian blur to the filled mask image to obtain a contour mask image containing edge pixels; and converting the edge pixels contained in the contour mask image into corresponding distance values ​​to obtain the distance-transformed image. The distance-transformed image is subjected to a second preprocessing step to obtain a second image; A second morphological operation is performed on the second image to calculate the number of vehicles in the original image. This includes: performing a preset number of erosion operations on the second image and marking connected components in the eroded image; grouping pixels with the same connected component marking into a single connected component; calculating multiple parameters for each connected component using a preset function, and determining whether the multiple parameters satisfy the vehicle counting condition; when each parameter satisfies the vehicle counting condition, determining the analysis result as recording the corresponding connected component; when any parameter does not satisfy the vehicle counting condition, determining the analysis result as not recording the corresponding connected component; and calculating the number of vehicles in the original image based on the analysis result.

2. The vehicle counting method as described in claim 1, characterized in that, The second morphological operation on the second image, calculating the number of vehicles in the original image, includes: Traverse all connected regions in the second image and calculate the number of vehicles in the original image.

3. The vehicle counting method as described in claim 1, characterized in that, The first preprocessing of the original image to obtain the first image includes: The original image is input into a pre-trained semantic segmentation model to obtain the segmentation result; The segmentation result is binarized to obtain the first image.

4. The vehicle counting method as described in claim 1, characterized in that, The process of performing a dilation operation on the first image to obtain a dilated image includes: Retrieve pre-created struct elements; The structuring element is used to sequentially compare and identify the first image to obtain the maximum pixel value in the image region covered by the structuring element. The gray value of the pixel at the center of the structuring element is then replaced with the maximum pixel value to obtain the dilated image.

5. The vehicle counting method as described in claim 4, characterized in that, The step of performing an erosion operation on the first image to obtain an eroded image includes: The structuring element is used to sequentially compare and identify the first image to obtain the minimum pixel value in the image area covered by the structuring element. The gray value of the pixel at the center of the structuring element is then replaced with the minimum pixel value to obtain the eroded image.

6. A vehicle counting device, characterized in that, The apparatus is used to implement the vehicle counting method as described in any one of claims 1 to 5, the apparatus comprising: An acquisition module is used to acquire the collected raw image and perform a first preprocessing on the raw image to obtain a first image, wherein the raw image contains multiple vehicles; The first operation module is used to perform a first morphological operation on the first image to obtain multiple contours in the first image, including: performing a dilation operation on the first image to obtain a dilated image; performing an erosion operation on the first image to obtain an eroded image; and subtracting the eroded image from the dilated image to obtain multiple contours in the first image. A determining module is used to determine a target contour based on the plurality of contours, wherein the target contour is a curve connecting all consecutive points with the same pixel value; A transformation module is used to perform distance transformation on the target contour to obtain a distance-transformed image, including: filling the interior of the target contour based on the pixel values ​​of the target contour; performing Gaussian blur on the filled mask image to obtain a contour mask image containing edge pixels; and converting the edge pixels contained in the contour mask image into corresponding distance values ​​to obtain the distance-transformed image. The preprocessing module is used to perform a second preprocessing on the distance-transformed image to obtain a second image; The second operation module performs a second morphological operation on the second image to calculate the number of vehicles in the original image.

7. An electronic device, characterized in that, The electronic device includes a processor and a memory, the processor being configured to implement the vehicle counting method as described in any one of claims 1 to 5 when executing a computer program stored in the memory.

8. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by the processor, it implements the vehicle counting method as described in any one of claims 1 to 5.

Images