A method and electronic device for identifying loaded vehicles based on moving target detection

Abstract

This application discloses a method and electronic device for identifying heavy vehicles based on moving target detection. The method includes: modeling the monitoring scene using the temporal median method; segmenting the image into blocks using the image to be detected and the background model; searching for the maximum structural similarity near the background blocks; classifying the image to be detected into foreground and background based on the maximum structural similarity, and recording the results using flags; employing a Fast Fourier Transform algorithm to improve the search speed for the maximum structural similarity; classifying the foreground and background to reduce the impact of camera shake; fitting the outer envelope of the detected vehicle with a rectangular outer contour; extracting the vehicle's length, width, and height features; and setting multiple thresholds to achieve vehicle identification even when part of the vehicle is located within the monitoring scene. Implementing the embodiments of this application can improve the accuracy of heavy vehicle identification.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and more specifically to a method and electronic device for identifying heavy vehicles based on moving target detection. Background Technology

[0002] The spatiotemporal distribution of vehicle loads on bridge structures is a crucial indicator of bridge load. Heavy vehicle loads, as a significant component of bridge structural loads, are a major factor contributing to bridge damage and even collapse. Therefore, monitoring heavy vehicle loads is of paramount importance.

[0003] Deploying image acquisition devices on bridge structures to obtain vehicle driving monitoring data and using computer vision technology to detect vehicles is a low-cost, high-precision, and easy-to-maintain method. It can be broadly divided into traditional moving target detection methods (inter-frame difference method, optical flow method, background subtraction method, etc.) and statistical learning-based methods.

[0004] Traditional moving target detection algorithms (background subtraction, inter-frame differencing, etc.) are simple to implement, fast, and versatile. When traffic is light, vehicle occlusion occurs infrequently, resulting in high detection accuracy. However, when applied to heavy vehicle detection, the significant camera shake caused by heavy vehicles can severely impact the method, making it difficult to distinguish between mutually occluding vehicles and leading to misclassification. Further refined schemes offer high detection accuracy but involve excessive computation. Summary of the Invention

[0005] This invention provides a method and electronic device for identifying loaded vehicles based on moving target detection, which can reduce the recognition error caused by background drift and thus improve the accuracy of loaded vehicle recognition under camera shake.

[0006] In a first aspect, embodiments of this application provide a method for identifying heavy vehicles based on moving target detection. This method, based on background subtraction, introduces a block-based approach to search for the maximum structural similarity, thus mitigating the impact of camera shake. It extracts the length, width, and height features of the vehicle by minimizing the envelope error between the model and the vehicle under the same perspective matrix, and then sets volume thresholds, length thresholds, width thresholds, and height thresholds to achieve heavy vehicle identification within a monitored scene. The method includes:

[0007] Step 1: Model the monitoring scene using the temporal median method, and segment the image using the image to be detected and the background model. Search for the maximum structural similarity near the background segment, and classify the foreground and background of the image to be detected based on the maximum structural similarity, and record it using a flag bit.

[0008] Step 2: Use the Fast Fourier Transform algorithm to improve the search speed of the maximum structural similarity, and perform foreground and background classification to reduce the impact of camera shake;

[0009] Step 3: Fit the outer envelope of the detected vehicle with the outer contour of the rectangular body, extract the length, width and height features of the vehicle, and set multiple sets of thresholds to enable vehicle recognition when the vehicle is partially located in the monitoring scene.

[0010] Secondly, embodiments of this application provide an electronic device, which includes: one or more processors and a memory;

[0011] The memory is coupled to the one or more processors, and the memory is used to store computer program code, the computer program code including computer instructions;

[0012] When the one or more processors execute the computer instructions, the electronic device performs the following operations:

[0013] Step 1: Model the monitoring scene using the temporal median method, and segment the image using the image to be detected and the background model. Search for the maximum structural similarity near the background segment, and classify the foreground and background of the image to be detected based on the maximum structural similarity, and record it using a flag bit.

[0014] Step 2: Use the Fast Fourier Transform algorithm to improve the search speed of the maximum structural similarity, and perform foreground and background classification to reduce the impact of camera shake;

[0015] Step 3: Fit the outer envelope of the detected vehicle with the outer contour of the rectangular body, extract the length, width and height features of the vehicle, and set multiple sets of thresholds to enable vehicle recognition when the vehicle is partially located in the monitoring scene.

[0016] Thirdly, embodiments of this application provide a computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the method steps provided by the first aspect or any implementation thereof of the embodiments of this application.

[0017] Fourthly, embodiments of this application provide a computer program product containing instructions that, when run on a computer or processor, cause the computer or processor to perform the method provided by the first aspect of the embodiments of this application or any possible implementation thereof.

[0018] As can be seen, the heavy vehicle recognition method and electronic device based on moving target detection provided in this application achieve accurate foreground / background classification by searching for the maximum structural similarity between the detected block and the surrounding background blocks within the monitored scene, reducing recognition errors caused by background drift, and thus improving the heavy vehicle recognition accuracy under camera shake. A vehicle contour 3D simplified model fitting method based on gradient descent is established. By minimizing the error between the vehicle contour envelope and the 3D model under the same perspective matrix, the optimal 3D fitting model of the vehicle envelope is obtained, and the size of the fitting model is the 3D spatial size of the vehicle being tested. The accuracy of the method is verified through simulated monitoring scenes and actual bridge heavy vehicle recognition experiments. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating a method for identifying loaded vehicles based on moving target detection, provided in an embodiment of this application.

[0020] Figure 2 This is a flowchart illustrating another method for identifying loaded vehicles based on moving target detection provided in an embodiment of this application;

[0021] Figure 3 This is an example of a schematic diagram of a surveillance video and background model provided in an embodiment of this application;

[0022] Figure 4 This is a scene diagram of the Changfeng Elevated Road provided in an embodiment of this application;

[0023] Figure 5 This is a schematic diagram illustrating the effect of a background subtraction method provided in an embodiment of this application;

[0024] Figure 6 This is a schematic diagram of a vehicle detection result based on structural similarity provided in an embodiment of this application;

[0025] Figure 7 This is a schematic diagram of a heavy vehicle identification result provided in an embodiment of this application;

[0026] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0027] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0028] The terms "first," "second," "third," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0029] To better understand the heavy vehicle identification method and electronic device based on moving target detection provided in this embodiment of the invention, it is possible to reduce the impact of camera shake and to identify heavy vehicles when part of the vehicle is located in the monitoring scene, thereby improving the identification accuracy.

[0030] The following describes a method for identifying loaded vehicles based on moving target detection, provided in an embodiment of this application. Please refer to [link to relevant documentation]. Figure 1 , Figure 1 This is a flowchart illustrating a method for identifying loaded vehicles based on moving target detection, provided in an embodiment of this application. Figure 1 As shown, the method for identifying loaded vehicles based on moving target detection may include:

[0031] Step 1: Model the monitoring scene using the temporal median method, and use the image to be detected and the background model to divide the image into blocks. By searching for the maximum structural similarity near the background block, the foreground and background are classified based on the maximum structural similarity of the image to be detected, and a flag is used to record the results.

[0032] Step 2: Use the Fast Fourier Transform algorithm to improve the search speed for maximum structural similarity and perform foreground and background classification to reduce the impact of camera shake;

[0033] Step 3: Fit the outer envelope of the detected vehicle with the outer contour of the rectangular body, extract the length, width and height features of the vehicle, and set multiple thresholds to enable vehicle recognition when part of the vehicle is located in the monitoring scene.

[0034] The heavy vehicle identification method provided in this application introduces a block-based approach based on background subtraction to search for the maximum structural similarity, thereby reducing the impact of camera shake. It extracts the length, width, and height features of the vehicle by minimizing the envelope error between the model and the vehicle under the same perspective matrix, and then sets volume threshold, length threshold, width threshold, and height threshold to achieve heavy vehicle identification in the monitoring scene.

[0035] The following is combined Figure 1 and Figure 2 This paper introduces a method for identifying loaded vehicles based on moving target detection, as provided in the embodiments of this application. Figure 2This is a flowchart illustrating another method for identifying loaded vehicles based on moving target detection provided in this application embodiment.

[0036] Step 1 can be specifically implemented as follows:

[0037] Step 1.1: Establish the background model using the time-domain median method:

[0038] V b (k)=median(V(i),i=k-jΔk,j∈[0,q-1])

[0039] That is, to achieve step S201, where V(i) is the grayscale matrix of the i-th frame of the video to be detected, V b (k) is the background grayscale matrix of the k-th frame of the video to be detected, q is the number of frames used for background modeling, Δk represents the sampling frame interval, and median represents taking the median of the sequence;

[0040] Step 1.2: Use background subtraction as the moving target detection method:

[0041] V f (k)=V(k)-V b (k)

[0042] In the formula, V f (k) is the foreground grayscale matrix of the k-th frame of the video to be detected;

[0043] Step 1.3: i and j are the pixel coordinates of the top-left corner of the block, and the structural similarity between the block to be detected and the background block is:

[0044]

[0045]

[0046]

[0047] In the formula s l (k,i,j), s c (k,i,j), s r (k,i,j) represent the brightness, contrast and correlation coefficient of the block respectively, u is the mean, σ is the standard deviation, cov is the covariance, and β1, β2 and β3 are small constants used to prevent systematic errors caused by the denominator being close to 0.

[0048] In some embodiments of this application, β2 and β3 satisfy β3 = β2 / 2.

[0049] Step 1.4: Divide the background model and the image to be detected into equal blocks. The side length of each block to be detected in the image is a. Use V(k,i,j), V b (k,i,j) uniquely identify the block to be detected and the background block, respectively, and the structural similarity is:

[0050] s(k,i,j)=s l (k,i,j)s c (k,i,j)s r (k,i,j)

[0051] Step 1.5: Simplify the calculation of the structural similarity s(k,i,j):

[0052]

[0053] In the formula, s l (k,i,j), s c (k,i,j), s r Since the range of values ​​for (k,i,j) is [-1,1], the range of values ​​for s(k,i,j) is also [-1,1].

[0054] Step 1.6: Search for the maximum structural similarity near the background block, and classify the block to be detected as foreground and background based on the maximum structural similarity:

[0055]

[0056] In the formula, V br (k,i,j) represents the background model V. b (k,ia,ja) to V b A rectangular region s of (k, i+a, j+a) m (k,i,j) is the maximum structural similarity between the block to be detected V(k,i,j) and all windows of equal size in the rectangular region; that is, step S202 is achieved.

[0057] Step 1.7: Based on the maximum structural similarity s m (k,i,j) performs foreground and background classification on the block to be detected V(k,i,j), and uses a flag bit to record the classification status of V(k,i,j).

[0058] That is, to implement step S203 in order to improve the monitoring accuracy under camera shake conditions.

[0059] Specifically, when s mWhen (k,i,j) is greater than the set threshold, V(k,i,j) is considered background and the flag is set to 1; otherwise, it is considered foreground and the flag is set to 0. Based on practical experience, the threshold is generally between 0.5 and 0.7. If a block is calculated not to belong to the background block, but its eight surrounding blocks all belong to the background block, then this block is highly likely to also belong to the background block; otherwise, it must contain information about the entire vehicle. Since the size of each block is relatively small, the probability of a block containing information about the entire vehicle is very low. Therefore, median filtering can be applied to the flag, and the final classification of each block to be detected can be determined based on the filtered result.

[0060] In some embodiments of this application, step 2 further includes:

[0061] Step 2.1: Input the block to be detected V(k,i,j) and the rectangular search region V of the background model. br (k,i,j);

[0062] Step 2.2: Use 0.01 2 0.03 2 0.03 2 / 2 Initialize small constants β1, β2, and β3, and initialize mean filter I with the same size as the block to be detected V(k,i,j);

[0063] Specifically, 0.01 2 0.03 2 0.03 2 / 2 is assigned to β1, β2, and β3 to initialize small constants. The filter constant can be specified by a function using 2 raised to the power of N, initializing the mean filter I of the same size as the block V(k,i,j). N can be a constant.

[0064] Step 2.3: Calculate the mean u[V(k,i,j)] of the block to be detected V(k,i,j);

[0065] Step 2.4: Apply the mean filter I to V br (k,i,j) are calculated within the rectangular search area V. br In (k,i,j), the mean value u[V] of each window of the same size as the block to be detected V(k,i,j) is obtained. br (k,i,j)];

[0066] Step 2.5: Calculate the variance σ[V(k,i,j)] of the block to be detected V(k,i,j) 2 And calculate the rectangular search area V br The variance σ[V] of each uniformly sized window (k,i,i) is denoted by a variable. br (k,i,j)] 2 V is processed by the mean filter I.br (k,i,j) 2 and subtract u[V br (k,i,j)] dot product u[V br (k,i,j)];

[0067] Step 2.6: Calculate the rectangular search area V br The covariance cov[V(k,i,j),V] of each equal-sized window of (k,i,i) with the block to be detected V(k,i,j) is [V(k,i,j),V]. br (k,i,j)];

[0068] Step 2.7: Process V using the block to be detected V(k,i,j) as a filter. br (k,i,j), subtract u[V(k,i,j)] dot product u[V] br (k,i,j)];

[0069] Step 2.8: Calculate V according to the structural similarity s(k,i,j) calculation formula. br The structural similarity s(k,i,i) between (k,i,i) and V(k,i,j);

[0070] in:

[0071]

[0072] Calculate the maximum value s in the set {s(k,i,j)} based on the structural similarity s(k,i,j). m (k,i,j) is used to separate the foreground and background to reduce the impact of camera shake.

[0073] In some embodiments of this application, step 3 further includes:

[0074] Step 3.1: Define a cuboid model as M(x,z,w,h,l) in the world coordinate system; where (x,z) describes the world coordinate system, with the z-axis along the road direction and the x-axis along the road's horizontal direction; (w,h,l) describes the width, height, and length of the cuboid model, respectively; and the image coordinate system uv represents the pixel coordinate system of the vehicle projected into the detection image, with the horizontal axis as the u-axis and the vertical downward axis as the v-axis.

[0075] Step 3.2: Based on the above definition, describe M. H The relationship between M and H:

[0076] M H =homo -1 {H[homo(M)]}

[0077] In the formula, H is the perspective matrix, representing the many-to-one mapping relationship between the 3D vehicle projection in the world coordinate system and the 2D vehicle projection in the image coordinate system, and M... H Let homo(x) be the projection of the cuboid model M onto the image coordinates under the mapping of the perspective matrix H, where homo(x) represents the transformation of the coordinates into homogeneous coordinates [mx]. T [m] T Let m be 1; homo -1 (x) represents the homogeneous coordinates [kx] T ,k] T The coordinates are converted to x, and the projection is obtained from the real model of the vehicle based on this transformation relationship;

[0078] Step 3.3: Based on the goal of finding the cuboid model that best describes the length, width, and height features of the 3D vehicle, the single sampling error of the ray is defined as follows:

[0079]

[0080] In the formula, mean(L) represents the center of the vehicle envelope L, mean(L) M The envelope L of the vehicle model's image coordinate projection under the perspective matrix H. M The center, d(μ(θ,L) M μ(θ,L)) represents the point μ(θ,L) M The Eulerian distance between ) and μ(θ,L); max(d(μ(θ,L) M ),μ(θ,L))) represents the maximum Eulerian distance across all intersection points; λ is a constant used to control the convergence of the two envelopes;

[0081] In some embodiments, L M This is determined by step 3.2 above.

[0082] Step 3.4: Based on the single sampling error e(L) M Based on (L,θ), the error between the two envelopes is calculated by uniform sampling, as follows:

[0083]

[0084] In the formula, Δθ is the angle interval between adjacent sampling rays;

[0085] Step 3.5: Based on the envelope error of the minimized model and the vehicle under the same perspective matrix, extract the length, width and height features of the vehicle. On the basis of the above steps, determine the heavy vehicle by setting the volume threshold, the length threshold, the width threshold and the height threshold respectively. When the extracted length, width and height dimensions and volume exceed the multiple sets of thresholds, it is identified as a heavy vehicle.

[0086] Specifically, minimizing the envelope error of the model and the vehicle under the same perspective matrix is ​​achieved in step S204. Furthermore, minimizing the envelope error of the model and the vehicle under the same perspective matrix is ​​achieved in step S205, whereby the optimally fitted model obtains the three-dimensional spatial dimensions.

[0087] In some embodiments, a loaded vehicle is determined by setting a volume threshold, a length threshold, a width threshold, and a height threshold, respectively. A vehicle is considered loaded if any one of these thresholds is met. These four thresholds effectively address situations where part of the vehicle is located within a monitored area.

[0088] Specifically, by setting the aforementioned threshold, heavy vehicles are identified, thus achieving step S206. Therefore, the heavy vehicle monitoring method is optimized through the aforementioned steps S201-S206.

[0089] The heavy vehicle recognition method based on moving target detection provided in this application achieves accurate foreground / background classification by searching for the maximum structural similarity between the target block and the surrounding background blocks within the monitored scene. This reduces recognition errors caused by background drift and thus improves the accuracy of heavy vehicle recognition under camera shake. A method for fitting a simplified 3D model of the vehicle contour based on gradient descent is established. By minimizing the error between the vehicle contour envelope and the 3D model under the same perspective matrix, the optimal 3D fitting model of the vehicle envelope is obtained, and the size of the fitting model is the 3D spatial size of the vehicle being tested. The accuracy of the method is verified through simulated monitoring scenes and actual bridge heavy vehicle recognition experiments.

[0090] The embodiments of this application are illustrated below.

[0091] like Figure 3 The following example illustrates the process of a heavy vehicle identification method based on moving target detection, using surveillance video from the Changfeng Elevated Bridge in Wuhan as an example. Figure 3 This is an example of a schematic diagram of a surveillance video and background model provided in an embodiment of this application. Figure 4 This is a scene diagram of the Changfeng Elevated Bridge provided in the embodiments of this application. Figure 3 (a) in the image is a video frame captured by a car while it is in motion. Figure 3 As shown in (a) in the figure, in Figure 4 The video was shot in a scene with a frame rate of 30 FPS, and the camera shake was obvious.

[0092] Step 1: Establish the background model using the temporal median filtering method, with 20 modeling frames and a frame interval of 10. Figure 3 (b) in the text is based on Figure 3The background model is established based on the image frame (a) in the diagram. The Fast Fourier Transform algorithm is used to significantly improve the search speed for maximum structural similarity, and this parameter is used as the basis for foreground / background classification to reduce the impact of camera shake. The outer envelope of the detected vehicle is fitted with a rectangular outer contour to extract the vehicle's length, width, and height features. Multiple thresholds are set to address situations where parts of the vehicle are located within the monitored scene.

[0093] A comparison was made between the traditional non-jitter background subtraction method and the structural similarity background subtraction method provided in the embodiments of this application, and the comparison results are as follows: Figure 5 As shown. Please refer to [the original text]. Figure 5 , Figure 5 This is a schematic diagram illustrating the effect of a background subtraction method provided in an embodiment of this application.

[0094] like Figure 5 As shown, two background subtraction methods were applied to the six groups of images to be detected. Figure 5 The first row shows six groups of images to be detected. The second row shows the results of processing each group of images using the traditional background subtraction method. The third row shows the results of processing each group of images using the structural similarity background subtraction method provided in this embodiment. It can be seen that comparing the results of the six groups of images after the two background subtraction methods, the structural similarity background subtraction method provided in this embodiment, when the camera is stable, achieves results essentially consistent with the traditional background subtraction method. This demonstrates that the structural similarity background subtraction method provided in this embodiment can achieve accurate background subtraction when the camera is stable.

[0095] The determination of whether the camera shakes or not can be made by setting a threshold. When the shaking amplitude exceeds the threshold, it is determined to be camera shaking, and when it does not exceed the threshold, it is determined to be camera not shaking.

[0096] Step 2: Comparison of traditional background subtraction and structural similarity background subtraction methods under stable camera conditions. Figure 5 As shown. Vehicle detection results based on structural similarity are as follows. Figure 6 As shown. Figure 6 This is a schematic diagram of a vehicle detection result based on structural similarity provided in an embodiment of this application.

[0097] like Figure 6 As shown, the first column of the image array represents 6 groups of images to be detected. The second column of the image array represents the background difference binary image of each image to be detected. The third column of the image array represents the structural similarity detection result of each image to be detected. The fourth column of the image array represents the binary image of the structural similarity detection result of each image to be detected.

[0098] Figure 6This can be a vehicle detection result based on structural similarity under camera shake conditions. In this embodiment, the background difference binary image is the detection result based on the traditional background subtraction method, and the structural similarity detection result binary image is the detection result based on structural similarity. Both the structural similarity detection result and the structural similarity detection result binary image employ a background subtraction method based on structural similarity to achieve accurate foreground / background classification, reduce recognition errors caused by background drift, and thus improve the accuracy of heavy vehicle recognition under camera shake conditions.

[0099] Step 3: Fit the outer envelope of the detected vehicle with the outer contour of the rectangular body, extract the length, width and height features of the vehicle, and set multiple thresholds to address the situation where part of the vehicle is located in the monitoring scene.

[0100] Please see Figure 7 , Figure 7 This is a schematic diagram of a heavy vehicle identification result provided in an embodiment of this application. Figure 7 This could be the result of heavy vehicle identification based on moving target detection, even under conditions of camera shake. Figure 7 It can be in Figure 6 Background recognition is performed based on the results, and heavy vehicle recognition based on moving target detection is realized.

[0101] like Figure 7 As shown, the first column in the image array represents the detection results for each of the six groups of images to be detected. Each image to be detected corresponds to... Figure 6 The binary image showing the structural similarity detection results is shown.

[0102] The second column in the image array represents the vehicle localization result for each image to be detected. The third column in the image array represents the heavy vehicle identification result for each image to be detected. The dashed rectangle is used for localization, and the length, width, and height identification results are represented by the fitted rectangular outer envelope. The solid outer envelope indicates that the identified length, width, and height features exceed the threshold and are judged as heavy vehicles.

[0103] Depend on Figure 6 and Figure 7 The identification results show that the heavy vehicle identification method based on structural similarity and vehicle length, width and height characteristics can effectively monitor heavy vehicles and has strong resistance to camera shake.

[0104] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. For example... Figure 8 As shown, the electronic device 200 may include at least:

[0105] At least one processor 501, at least one network interface 504, user interface 503, memory 505, and at least one communication bus 502.

[0106] The communication bus 502 is used to enable communication between these components.

[0107] The user interface 503 may include one or more of a display screen and a camera. Optionally, the user interface 503 may also include a standard wired interface and a wireless interface.

[0108] The network interface 504 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0109] The processor 501 may include one or more processing cores. The processor 501 connects to various parts of the electronic device 200 using various interfaces and lines, and performs various functions of the electronic device 200 and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 505, and by calling data stored in the memory 505.

[0110] Optionally, the processor 501 can be implemented using at least one of the following hardware forms: Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 501 can integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem can also be implemented as a separate chip without being integrated into the processor 501.

[0111] The memory 505 may include random access memory (RAM) or read-only memory. Optionally, the memory 505 may include a non-transitory computer-readable storage medium. The memory 505 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 505 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 505 may also be at least one storage device located remotely from the aforementioned processor 501. Figure 8 As shown, the memory 505, which serves as a computer storage medium, may include an operating system, a network communication module, and a user interface module.

[0112] exist Figure 8 In the electronic device 200 shown, the user interface 503 is mainly used to provide an input interface for the user and to obtain the user input data; while the processor 501 can be used to call the application program stored in the memory 505 and specifically execute the program operation.

[0113] In some embodiments of this application, when the one or more processors execute the computer instructions, the electronic device causes the following operations to be performed:

[0114] Step 1: Model the monitoring scene using the temporal median method, and segment the image using the image to be detected and the background model. Search for the maximum structural similarity near the background segment, and classify the foreground and background of the image to be detected based on the maximum structural similarity, and record it using a flag bit.

[0115] Step 2: Use the Fast Fourier Transform algorithm to improve the search speed of the maximum structural similarity, and perform foreground and background classification to reduce the impact of camera shake;

[0116] Step 3: Fit the outer envelope of the detected vehicle with the outer contour of the rectangular body, extract the length, width and height features of the vehicle, and set multiple sets of thresholds to enable vehicle recognition when the vehicle is partially located in the monitoring scene.

[0117] The electronic device provided in this application achieves accurate foreground / background classification by searching for the maximum structural similarity between the target block and the surrounding background blocks within the monitored scene. This reduces recognition errors caused by background drift and improves the accuracy of heavy vehicle recognition under camera shake. A method for fitting a simplified 3D model of the vehicle contour based on gradient descent is established. By minimizing the error between the vehicle contour envelope and the 3D model under the same perspective matrix, the optimal 3D fitting model of the vehicle envelope is obtained, and the size of the fitting model is the 3D spatial size of the vehicle being tested. The accuracy of the method is verified through simulated monitoring scenes and actual bridge heavy vehicle recognition experiments.

[0118] This application also provides a computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform the above-described instructions. Figure 1 One or more steps in the illustrated embodiment. If the constituent modules of the above-described electronic device are implemented as software functional units and sold or used as independent products, they can be stored in the computer-readable storage medium.

[0119] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital versatile discs (DVDs)), or semiconductor media (e.g., solid-state drives (SSDs)).

[0120] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The aforementioned storage medium includes various media capable of storing program code, such as Read Only Memory (ROM), Random Access Memory (RAM), magnetic disks, or optical disks. Unless otherwise specified, the technical features of this embodiment and its implementation schemes can be combined arbitrarily.

[0121] The embodiments described above are merely preferred embodiments of this application and are not intended to limit the scope of this application. Any modifications and improvements made by those skilled in the art to the technical solutions of this application without departing from the spirit of this application should fall within the protection scope defined by the claims of this application.

[0122] The methods and electronic devices disclosed in the embodiments of the present invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A heavy vehicle identification method based on moving object detection, characterized in that, The method weakens the influence of camera shaking by introducing the block idea on the basis of background subtraction method and searching the maximum structural similarity; the vehicle length-width-height features are extracted by minimizing the model and the envelope error of the vehicle under the same perspective matrix, and then the volume threshold, length threshold, width threshold and height threshold are set to realize the heavy vehicle recognition in the monitoring scene, and the method comprises: Step 1: the monitoring scene is modeled by using the time domain median method, and the image blocks are divided by using the to-be-detected image and the background model; the maximum structural similarity is searched in the vicinity of the background block, the to-be-detected block is separated from the background by using the maximum structural similarity, and a flag bit is recorded; Step 2: the fast Fourier transform algorithm is used to improve the search speed of the maximum structural similarity, the foreground and the background are separated to reduce the influence of camera shaking; Step 3: the outer envelope of the cuboid is fitted to detect the outer envelope of the vehicle, the length-width-height features of the vehicle are extracted, and multiple threshold values are set to realize the vehicle recognition when the vehicle is partially located in the monitoring scene; The step 1 further comprises: The background model is established by using the time domain median method: V b (k) = median(V(i), i = k - jAk, j e [0, q - 1]) wherein V(i) is the gray matrix of the i-th picture of the video to be detected, V b (k) is the background gray matrix of the k-th picture of the video to be detected, q is the number of frames used for background modeling, Δk represents the sampling frame number interval, and median indicates taking the median of the sequence. The background subtraction method is used as the motion target detection method: V f (k) = V(k) - V b (k) In the formula, V f (k) is the foreground gray matrix of the kth picture of the video to be detected; Wherein, i, j are the pixel coordinates of the upper left corner of the block, the structural similarity between the to-be-detected block and the background block is: ； ； ； where s l (k,i,j), s c (k,i,j), s r (k,i,j) respectively represent the block's luminance, contrast and correlation coefficient, u is the mean, σ is the standard deviation, cov is the covariance, β1, β2, β3 are small constants to prevent system error caused by denominator close to 0; The background model and the picture to be detected are divided into equal blocks, and the side length of each block of the picture to be detected is a. V(k,i,j) and V(k,i,j) uniquely mark the block to be detected and the background block, respectively, and the structural similarity is: b (k,i,j) respectively uniquely mark the block to be detected and the background block, and the structural similarity is: s(k,i,j) = s l (k,i,j) s c (k,i,j) s r (k,i,j) The calculation of the structural similarity s(k, i, j) is simplified: ； In the formula, s l (k,i,j), s c (k,i,j), s r (k,i,j) are all in the range of [-1, 1], the range of s(k,i,j) is also [-1, 1]. The maximum structural similarity is searched in the vicinity of the background block, and the to-be-detected block is classified into foreground and background by using the maximum structural similarity: ； wherein V br (k,i,j) is the background model V b (k,i-a,j-a) to V b (k,i+a,j+a) of the rectangular region, s m (k,i,j) is the maximum structural similarity of the to-be-detected block V(k,i,j) and all equal-size windows of the rectangular region. According to the maximum structural similarity s m The (k, i, j) pair classifies the to-be-detected block V(k, i, j) into foreground and background, and adopts a flag bit to record the classification state of V(k, i, j). The step 2 further comprises: Input the block V(k,i,j) to be detected and the background model rectangular search region V br (k,i,j) with 0.01 2 , 0.03 2 , 0.03 2 Initialize small constants β1, β2, β3, and initialize the mean filter I with the same size as the block V(k,i,j) to be detected. The mean value u[V(k, i, j)] of the to-be-detected block V(k, i, j) is calculated; with the mean filter I on V br (k,i,j) in the rectangular search region V br (k,i,j), obtaining the mean u[V br (k,i,j)] of each window of the same size as the block to be detected V(k,i,j). Calculate the variance σ[V(k,i,j)] of the block to be detected V(k,i,j) 2 And calculate the rectangular search area V br The variance σ[V] of each uniformly sized window (k,i,i) is denoted by a variable. br (k,i,j)] 2 V is processed by the mean filter I. br (k,i,j) 2 and subtract u[V br (k,i,j)] dot product u[V br (k,i,j)]; computing the rectangular search region V br each equal-sized window of (k, i, i) with the covariance cov[V(k, i, j), Vbr(k, i, j)] of the block V(k, i, j) to be detected; by processing said to be detected patch V(k,i,j) as a filter V br (k,i,j), subtracting u[V(k,i,j)] point multiplication u[V br (k,i,j)]. According to the structure similarity s(k,i,j) calculation formula, calculate V br (k,i,i) and the structure similarity s(k,i,j) of V(k,i,j); The maximum value s(k,i,j) in the set {s(k,i,j)} is calculated according to the structural similarity s(k,i,j) m (k,i,j) is calculated, and foreground / background separation is performed to reduce the influence of camera shake.

2. The heavy vehicle identification method based on moving object detection according to claim 1, wherein, The β2, β3 satisfy β3 = β2 / 2.

3. The heavy vehicle identification method based on moving object detection according to any one of claims 1-2, wherein, The step 3 further comprises: The cuboid model is defined in the world coordinate system as M(x, z, w, h, l); wherein (x, z) describes the world coordinate system, z axis along the road direction, x axis in the road transverse direction, (w, h, l) respectively describes the width, height and length of the cuboid model, and the picture coordinate system uv represents the pixel coordinate system of the vehicle projection in the detection image, with the horizontal as the u axis and the vertical downward as the v axis; Based on the simulation of the above definition, M H , M, H relationship: M H = homo -1 In the formula {H[homo(M)]}, H is a perspective matrix, representing a many-to-one mapping relationship of the three-dimensional vehicle in the world coordinate system and the two-dimensional vehicle projection in the picture coordinate system, M H is the projection of the cuboid model M in the picture coordinate under the mapping of the perspective matrix H, homo(x) represents converting coordinates into homogeneous coordinates [mx T , m] T , taking m as 1; homo -1 (x) represents converting homogeneous coordinates [kx T , k] T into coordinates x, and the projection is obtained from the real model of the vehicle according to the transformation relationship. Based on the purpose of finding the cuboid model best describing the three-dimensional vehicle length-width-height features, the single sampling error of the ray is defined, and the specific content is as follows: ； In the formula, mean(L) represents the center of the vehicle envelope L, mean(L) M The envelope L of the vehicle model's image coordinate projection under the perspective matrix H. M The center, d(μ(θ,L) M μ(θ,L)) represents the point μ(θ,L) M The Eulerian distance between ) and μ(θ,L); max(d(μ(θ,L) M ),μ(θ,L))) represents the maximum Eulerian distance across all intersection points; λ is a constant used to control the convergence of the two envelopes; based on the single-sampling error e(L M On the basis of the single-sampling error e(L , θ), the error of the two envelope lines is calculated by performing uniform sampling, specifically as follows: ； In the formula, Δθ is the adjacent sampling ray angle interval; According to the minimum model and the envelope error of the vehicle under the same perspective matrix, the vehicle length-width-height size features are extracted, and on the basis of the above steps, the heavy vehicle is determined by setting the volume threshold, the length threshold, the width threshold and the height threshold respectively; when the extracted length-width-height size and volume exceed the multiple threshold values, the heavy vehicle is determined.

4. An electronic device, comprising: The electronic device comprises one or more processors, a memory; The memory is coupled with the one or more processors, and is configured to store computer program codes, the computer program codes comprising computer instructions; when the one or more processors execute the computer instructions, the electronic device is caused to perform the following operations: Step 1: modeling the monitoring scene by using a time domain median method, and performing picture blocking by using a to-be-detected picture and a background model, searching for a maximum structural similarity near a background block, classifying a to-be-detected block according to the maximum structural similarity, and recording by using a flag bit; Step 2: using a fast Fourier transform algorithm to improve the search speed of the maximum structural similarity, and performing the foreground and background classification to reduce the influence of camera shaking; Step 3: fitting an outer contour of a cuboid to detect an outer envelope of a vehicle, extracting a length-width-height feature of the vehicle, and setting multiple threshold values to realize the vehicle recognition when the vehicle is partially located in the monitoring scene; The processor is specifically configured to invoke the program code to perform the following operations: A background model is established by using a time domain median method: V b (k) = median(V(i), i = k - jAk, j e [0, q - 1]) wherein V(i) is the gray matrix of the i-th picture of the video to be detected, V b (k) is the background gray matrix of the k-th picture of the video to be detected, q is the number of frames used for background modeling, Δk represents the sampling frame number interval, and median indicates taking the median of the sequence. A background subtraction method is used as a moving target detection method: V f (k) = V(k) - V b (k) In the formula, V f (k) is the foreground gray matrix of the kth picture of the video to be detected; Wherein, i, j are pixel coordinates of the upper left corner of the block, a structural similarity between the to-be-detected block and the background block is: ； ； ； where s l (k,i,j), s c (k,i,j), s r (k,i,j) respectively represent the block's luminance, contrast and correlation coefficient, u is the mean, σ is the standard deviation, cov is the covariance, β1, β2, β3 are small constants to prevent system error caused by denominator close to 0; The background model and the picture to be detected are divided into equal blocks, and the side length of each block of the picture to be detected is a. V(k,i,j) and V(k,i,j) uniquely mark the block to be detected and the background block, respectively, and the structural similarity is: b (k,i,j) respectively uniquely mark the block to be detected and the background block, and the structural similarity is: s(k, i, j) = s l (k, i, j) s c (k, i, j) s r (k, i, j) The calculation of the structural similarity s(k, i, j) is simplified: ； In the formula, s l (k,i,j), s c (k,i,j), s r (k,i,j) are all in the range of [-1, 1], the range of s(k,i,j) is also [-1, 1]. The maximum structural similarity is searched near the background block, and the to-be-detected block is classified according to the maximum structural similarity; ； wherein V br (k,i,j) is the background model V b (k,i-a,j-a) to V b (k,i+a,j+a) of the rectangular region, s m (k,i,j) is the maximum structural similarity of the to-be-detected block V(k,i,j) and all equal-size windows of the rectangular region. According to the maximum structural similarity s m The (k, i, j) pair classifies the to-be-detected block V(k, i, j) into foreground and background, and adopts a flag bit to record the classification state of V(k, i, j). The processor is specifically configured to invoke the program code to perform the following operations: Input the block V(k,i,j) to be detected and the background model rectangular search region V br (k,i,j) Initialize small constants β1, β2, β3 with 0.01 2 , 0.03 2 , 0.03 2 / 2, initialize mean filter I with the same size as the block V(k,i,j) to be detected; The mean value u[V(k, i, j)] of the to-be-detected block V(k, i, j) is calculated. with the mean filter I on V br (k,i,j) in the rectangular search region V br (k,i,j), obtaining the mean u[V br (k,i,j)] of each window of the same size as the block to be detected V(k,i,j). Calculate the variance σ[V(k,i,j)] of the block to be detected V(k,i,j) 2 And calculate the rectangular search area V br The variance σ[V] of each uniformly sized window (k,i,i) is denoted by a variable. br (k,i,j)] 2 V is processed by the mean filter I. br (k,i,j) 2 and subtract u[V br (k,i,j)] dot product u[V br (k,i,j)]; computing the rectangular search region V br Each equal-sized window of (k, i, j) is associated with the covariance cov[V(k, i, j), V br (k, i, j)] of the block V(k, i, j) to be detected. by processing said to be detected patch V(k,i,j) as a filter V br (k,i,j), subtracting u[V(k,i,j)] point multiplication u[V br (k,i,j)]. According to the structure similarity s(k, i, j) calculation formula, calculate V br (k, i, i) and the structure similarity s(k, i, j) of V(k, i, j); The maximum value s(k,i,j) in the set {s(k,i,j)} is calculated according to the structural similarity s(k,i,j) m (k,i,j) is performed to reduce the influence of camera shake.

5. The electronic device of claim 4, wherein, The β2, β3 satisfy β3 = β2 / 2.

6. The electronic device of any of claims 4-5, wherein, The processor is specifically configured to invoke the program code to perform the following operations: A cuboid model is defined in a world coordinate system as M(x, z, w, h, l); wherein (x, z) describes the world coordinate system, z axis is along the road direction, x axis is the road transverse direction, (w, h, l) respectively describe the width, height and length of the cuboid model, and a picture coordinate system uv represents a pixel coordinate system of the vehicle projected in the detection picture, with the horizontal direction as the u axis and the vertical direction downward as the v axis; Based on the simulation of the above definition, the relationship of M H , M, H is described: M H = homo -1 {H[homo(M)]} where H is a perspective matrix representing a many-to-one mapping relationship of the three-dimensional vehicle in the world coordinate system to the two-dimensional vehicle projection in the picture coordinate system, M H is a projection of the cuboid model M in the picture coordinate under the mapping of the perspective matrix H, homo(x) represents converting a coordinate into homogeneous coordinates [mx T ,m] T , taking m as 1; homo -1 (x) represents converting homogeneous coordinates [kx T ,k] T into a coordinate x, and the projection is obtained from the real model of the vehicle according to the transformation relationship. Based on the purpose of finding a cuboid model best describing the length-width-height features of the three-dimensional vehicle, a single sampling error of a ray is defined as follows: ； In the formula, mean(L) represents the center of the vehicle envelope L, mean(L) M The envelope L of the vehicle model's image coordinate projection under the perspective matrix H. M The center, d(μ(θ,L) M μ(θ,L)) represents the point μ(θ,L) M The Eulerian distance between ) and μ(θ,L); max(d(μ(θ,L) M ),μ(θ,L))) represents the maximum Eulerian distance across all intersection points; λ is a constant used to control the convergence of the two envelopes; based on the single-sampling error e(L M On the basis of the single-sampling error e(L , L, θ), the error of the two envelope lines is calculated by performing uniform sampling, specifically as follows: ； In the formula, Δθ is the interval of the adjacent sampling ray angles; According to the minimization model and the envelope error of the vehicle under the same perspective matrix, the length-width-height size features of the vehicle are extracted, and on the basis of the above steps, a heavy vehicle is determined by respectively setting a volume threshold value, a length threshold value, a width threshold value and a height threshold value; when the extracted length-width-height size and volume exceed the multiple threshold values, the heavy vehicle is determined.

Images