Image rectification method, illegal parking detection method and device

Abstract

This application relates to the field of computer vision, and more particularly to an image correction method. The method involves calculating a correction matrix by comparing the pixel coordinates of a first and a second identifier region; then transforming the pixel coordinates of the first region of interest (ROI) using the correction matrix to obtain the first pixel coordinates; finally, generating a second ROI on the second image based on the first pixel coordinates. The first and second identifier regions are regions formed on the first and second images respectively by the same identifier. By comparing the pixel coordinates of the first and second identifier regions, the offset (correction matrix) between them can be obtained. This offset is used to compensate for the first ROI to obtain the second ROI. Replacing the first ROI with the second ROI completes the correction, thus solving the problem of ROI offset.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image correction method, a method and apparatus for detecting illegal parking. Background Technology

[0002] By setting image acquisition devices in preset locations, images can be continuously acquired over extended periods, such as for hydrological monitoring, road surveillance, and indoor monitoring, enabling real-time analysis of the monitored area. In some applications, the user's interest may not lie in the entire image. For example, when monitoring water levels, only the water gauge area needs analysis; similarly, when monitoring illegal parking, only the illegal parking area needs analysis. Current technology typically involves the user designating a Region of Interest (ROI) on the image, and subsequent analysis only examines the image within that ROI. However, it has been found that image acquisition devices can shift or deflect due to their mechanical structure or external interference, causing the pre-defined ROI to shift as well. In this case, the image within the ROI is no longer the user's area of ​​interest, rendering further analysis meaningless. Summary of the Invention

[0003] This application provides an image correction method that solves the problem of ROI offset.

[0004] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows:

[0005] In a first aspect embodiment, the image correction method is based on a first image and a second image acquired at the same location by the same image acquisition device. The first image has a first region of interest and also includes a first identifier region. The second image includes a second identifier region, wherein the first identifier region and the second identifier region are regions formed by the same identifier on the first image and the second image, respectively. The image correction method includes:

[0006] Obtain the pixel coordinates of the first identification region and the second identification region, compare the pixel coordinates of the first identification region and the pixel coordinates of the second identification region, and obtain the correction matrix;

[0007] Obtain the pixel coordinates of the first region of interest, and transform the pixel coordinates of the first region of interest through the correction matrix to obtain the second region of interest;

[0008] The second region of interest replaces the first region of interest, thus completing the correction.

[0009] This application compares the pixel coordinates of the first and second identifier regions to calculate the correction matrix. Then, it transforms the pixel coordinates of the first region of interest (ROI) using the correction matrix to obtain the first pixel coordinates. Finally, it generates a second ROI on the second image based on the first coordinates, and replaces the first ROI with the second ROI to complete the ROI correction. The first and second identifier regions are regions formed on the first and second images respectively by the same identifier. By comparing the pixel coordinates of the first and second identifier regions, the offset between them (correction matrix) can be obtained. This offset is used to compensate for the first ROI to obtain the second ROI. Replacing the first ROI with the second ROI completes the ROI correction, thus solving the ROI offset problem.

[0010] In some implementations, the correction matrix is:

[0011]

[0012] Wherein, Δt is the change in pitch angle of the image acquisition device when acquiring the first image and the second image; Δp is the change in heading angle of the image acquisition device when acquiring the first image and the second image.

[0013] In some implementations, the correction matrix is ​​calculated in the following manner:

[0014] The imaging formula of the image acquisition device is:

[0015]

[0016] Where (X, Y, Z) are world coordinates; (x, y) are pixel coordinates; This is the intrinsic parameter matrix; Let be the rotation matrix, t be the pitch angle, and p be the yaw angle;

[0017] Let K denote the intrinsic parameter matrix and T denote the rotation matrix. Transforming the imaging formula, we obtain the reconstruction formula:

[0018]

[0019] The world coordinates of the identifier are set as (X). i Y i Z i ), and mark the pixel coordinates of the first identified region as (x i y i The pixel coordinates of the second identified region are marked as (x′). i y′ iLet T1 be the rotation matrix used to acquire the first image, and T2 be the rotation matrix used to acquire the second image; let (X... i Y i Z i ), (x i y i Substituting T1 into the reduction formula, we get:

[0020]

[0021] (X) i Y i Z i ), (x′ i y′ i Substituting T2 into the imaging formula, we get:

[0022]

[0023] Substituting the reduction formula into the imaging formula yields the target formula:

[0024]

[0025] in,

[0026]

[0027] Construct the objective function based on the stated objective formula:

[0028]

[0029] The optimal solutions for Δt and Δp are obtained through a nonlinear optimization algorithm, and then the correction matrix is ​​calculated.

[0030] In some implementations, the first pixel coordinates are calculated in the following manner:

[0031] Let the pixel coordinates of the first region of interest be (x j y j ), will (x j y j Substituting the correction matrix into the target formula, we obtain the first pixel coordinate (x'). j y' j ):

[0032]

[0033] In some implementations, when both the first and second identification regions contain multiple points, the correction matrix is ​​calculated using the following steps:

[0034] S1351. Match the points in the first identification area with the points in the second identification area to obtain multiple pairs of matching points;

[0035] S1352. Substitute the pixel coordinates of the multiple pairs of matching points into the objective function to solve for multiple correction matrices, and select the correction matrix with the smallest loss value from the multiple correction matrices.

[0036] S1353. Substitute the pixel coordinates of the points in the first identification region among the multiple pairs of matched points and the correction matrix with the smallest loss value into the target formula to obtain multiple reference points;

[0037] S1354. Calculate the distance between each reference point and the corresponding point in the second identification area, select the reference points that are less than the distance threshold, and retain the points in the second identification area corresponding to the reference points that are less than the distance threshold.

[0038] S1355. Rematch the points retained in the second identification area with the points in the first identification area, and repeat step S1352 to obtain a better correction matrix.

[0039] In some embodiments, the markings include: signs, signs, traffic markings, ground markings, parking lines, and shorelines.

[0040] In a second aspect of the embodiment, this application provides a method for detecting illegal parking, comprising:

[0041] The second region of interest is determined using the image correction method described above, and is used as the new region of interest.

[0042] The second image is identified to obtain vehicle information within the second region of interest, which is recorded as the first vehicle information.

[0043] Acquire a third image, which is the next frame of the second image; identify the third image to obtain vehicle information within the region of interest, and record it as the second vehicle information.

[0044] The second vehicle information is compared with the first vehicle information. If they are the same, the consecutive frames are incremented by one; if they are different, the first vehicle information is deleted and the second vehicle information is recorded.

[0045] This process continues, acquiring subsequent consecutive frames. When the number of consecutive frames exceeds a preset value, it is determined that the vehicle has engaged in illegal parking.

[0046] In some embodiments, the illegal parking detection method further includes: setting triggering conditions for the image correction method; the triggering conditions include at least one of the following: a preset time interval, the image acquisition device moving to a preset position, and receiving a correction command; when any of the triggering conditions is met, image correction is performed.

[0047] In a third aspect of the embodiment, this application provides an image correction device, based on a first image and a second image acquired sequentially at the same location by the same image acquisition device. A first region of interest is defined on the first image, and the first image also includes a first identifier region. The second image includes a second identifier region, wherein the first identifier region and the second identifier region are regions formed by the same identifier on the first image and the second image respectively. The image correction device includes:

[0048] The receiving module is used to obtain the pixel coordinates of the first identified region and the pixel coordinates of the first region of interest;

[0049] The recognition module is used to recognize the second image and obtain the pixel coordinates of the second identified region;

[0050] The first data processing module is used to calculate the correction matrix;

[0051] The second data processing module is used to transform the pixel coordinates of the first region of interest through the correction matrix to obtain the first pixel coordinates;

[0052] The generation module is used to generate a second region of interest on the second image based on the first pixel coordinates, and replace the first region of interest with it.

[0053] In a fourth aspect embodiment, this application provides an illegal parking detection device, comprising:

[0054] The aforementioned image correction device;

[0055] The first vehicle detection module is used to identify the second image and obtain vehicle information within the second region of interest, which is denoted as the first vehicle information.

[0056] The second vehicle detection module is used to acquire a third image, which is the next frame of the second image, identify the third image, and obtain vehicle information in the third region of interest, which is denoted as the second vehicle information.

[0057] The comparison module is used to compare the second vehicle information with the first vehicle information. If they are the same, the consecutive frames are incremented by one; if they are different, the first vehicle information is deleted and the second vehicle information is recorded.

[0058] The judgment module is used to determine whether the vehicle has committed a traffic violation. When the number of consecutive frames exceeds a preset value, it is determined that the vehicle has committed an illegal parking violation.

[0059] Additional aspects and advantages of embodiments of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of embodiments of this application. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] The methods, systems, and / or procedures shown in the accompanying drawings will be further described with reference to exemplary embodiments. These exemplary embodiments will be described in detail with reference to the drawings. These exemplary embodiments are non-limiting exemplary embodiments, wherein example figures represent similar mechanisms in the various views of the drawings.

[0062] Figure 1 This is a flowchart of an image correction method in one embodiment;

[0063] Figure 2 This is a flowchart illustrating the calculation of the correction matrix in an image correction method according to one embodiment;

[0064] Figure 3 This is a flowchart of an embodiment of a method for detecting illegal parking;

[0065] Figure 4 This is a schematic diagram of the structure of an image correction device in one embodiment;

[0066] Figure 5 This is a schematic diagram of the structure of a parking violation detection device in one embodiment;

[0067] Explanation of reference numerals in the attached figures

[0068] 10 - Image correction device;

[0069] 11-Receiving module; 12-Identification module; 13-First data processing module; 14-Second data processing module; 15-Generation module;

[0070] 20 - Illegal parking detection device;

[0071] 21-First vehicle detection module; 22-Second vehicle detection module; 23-Comparison module; 24-Judgment module. Detailed Implementation

[0072] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0073] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0074] In the description of this invention, it should be noted that the terms "first," "second," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0075] When image acquisition equipment is used for scenarios such as hydrological monitoring or traffic violation detection, users set the target area in the monitored scene as the Region of Interest (ROI) on the image. Subsequent analysis only examines the image within the ROI. For example, when monitoring water levels, the water gauge area is set as the ROI; when monitoring illegal parking, the violation area is set as the ROI. However, over time, due to aging of the image acquisition equipment itself or changes in the installation environment, the image acquisition equipment may shift, and the acquired images will also shift. Since the ROI is manually set by the user, after the image shifts, the ROI will also shift accordingly. At this point, the image within the ROI is no longer the image of the target area, and subsequent analysis will only yield incorrect results.

[0076] This application solves the ROI offset problem by using the markers in the monitoring area as a reference to calculate the correction matrix and then recorrecting the ROI.

[0077] Please see Figure 1 In the first aspect of the embodiment, this application provides an image correction method based on a first image and a second image acquired by the same image acquisition device at the same location, and a first identifier region and a second identifier region formed on the first image and the second image respectively by the same identifier, and a first region of interest on the first image.

[0078] It should be noted that the aforementioned markings include signs, markers, traffic markings, ground markings, parking lines, shorelines, etc. in the monitoring scenario.

[0079] The image correction method includes:

[0080] S11. Obtain the pixel coordinates of the first identified region and the pixel coordinates of the first region of interest.

[0081] For example, if the aforementioned sign is a rectangular road sign on a road, then the first sign area is a quadrilateral area. The pixel coordinates of the first sign area can be represented by the pixel coordinates of any one or more points in the quadrilateral area. Preferably, they can be represented by the pixel coordinates of the centroid or vertex of the quadrilateral area.

[0082] In some specific implementations, the first identified region is obtained using the YOLOv5 image recognition algorithm. First, the YOLOv5 algorithm is trained: the video image is manually labeled, marking the identified regions on the image. After labeling, the YOLOv5 algorithm is used to detect the image, and the labeled regions are used for training to obtain the optimal detection model. Then, the YOLOv5 algorithm is used to recognize the first image to find the first identified region. The first region of interest is drawn manually.

[0083] After obtaining the pixel coordinates of the first identified region and the first region of interest, and after manual verification, they are written into the configuration file. In subsequent image correction, the data in the configuration file is used directly as the basis to improve the accuracy of image correction.

[0084] S12. Recognize the second image and obtain the pixel coordinates of the second identified region.

[0085] S13. Calculate the correction matrix, including:

[0086] S131. Transform the imaging formula into a restoration formula, and substitute the world coordinates of the marker and the pixel coordinates of the first marker area into the restoration formula.

[0087] Specifically:

[0088] The imaging formula of the image acquisition device is:

[0089]

[0090] Where (X, Y, Z) are world coordinates; (x, y) are pixel coordinates; This refers to the intrinsic parameter matrix of the aforementioned image acquisition device; The rotation matrix is ​​denoted by t, where t is the pitch angle and p is the yaw angle. The offset of the ROI region is caused by the angular offset of the image acquisition device, mainly due to changes in the yaw and pitch angles of the image acquisition device, while the roll angle remains unchanged. Therefore, the imaging formula given in this method does not include the rotation matrix of the roll angle.

[0091] internal parameter matrix Let K be the rotation matrix. Let T be the image quality formula, then the imaging formula can be simplified to:

[0092]

[0093] The restored matrix can be obtained after deformation as follows:

[0094]

[0095] The world coordinates of the identifier are (X i Y i Z i ), and mark the pixel coordinates of the first identified region as (x i y i ), and mark the pixel coordinates of the second identifier region as (x' i y' i Let T1 be the rotation matrix when acquiring the first image and T2 be the rotation matrix when acquiring the second image.

[0096] (X) i Y i Z i ), (x i y i Substituting T1 into the reduction formula, we get:

[0097]

[0098] S132. Substitute the world coordinates of the marker and the pixel coordinates of the second marker area into the imaging formula.

[0099] Specifically, (X) i Y i Z i ), (x' i y' i Substituting T2 into the imaging formula, we get:

[0100]

[0101] S133. Substitute the restoration formula into the imaging formula to obtain the target formula.

[0102] Specifically, substituting Equation 1 into Equation 2 yields:

[0103]

[0104] in,

[0105] Where Δt represents the change in pitch angle of the image acquisition device when acquiring the first image and the second image, and Δp represents the change in heading angle of the image acquisition device when acquiring the first image and the second image.

[0106] S134. Find the correction matrix.

[0107] Specifically, the objective function is constructed according to Equation 3:

[0108]

[0109] The optimal solutions for Δt and Δp are obtained using a nonlinear optimization algorithm, and then substituted into T2*T1. -1 Find the correction matrix:

[0110]

[0111] In some embodiments, to obtain a better correction matrix, the following steps may also be included:

[0112] S135. Find a better correction matrix.

[0113] S1351. Match the points in the first and second identification regions to obtain multiple pairs of matching points;

[0114] S1352. Substitute the pixel coordinates of multiple pairs of matching points into the objective function 4 to solve for multiple correction matrices, and select the correction matrix with the smallest loss value from the multiple correction matrices.

[0115] S1353. Substitute the pixel coordinates of the points in the first identified region among multiple pairs of matched points and the correction matrix with the smallest loss value into the target formula 3 to obtain multiple reference points;

[0116] S1354. Calculate the distance between each reference point and the corresponding point in the second identification area, select the reference points that are less than the distance threshold, retain the points in the second identification area corresponding to the reference points that are less than the distance threshold, and delete the rest.

[0117] S1355. Rematch the points retained in the second identification area with the points in the first identification area, and repeat step S1352 to select the correction matrix with the smallest loss value, which is the better correction matrix.

[0118] The following explanation uses a rectangular road sign as an example to illustrate step S135 above:

[0119] When both the first and second identification regions contain multiple points, matching the points in the first and second identification regions yields multiple pairs of matching points. Specifically, the first identification region corresponding to the rectangular road sign is a quadrilateral region, with the top-left vertex of the first identification region being i1, and its pixel coordinates being (x...). i1 y i1 The top left corner of the rectangular road sign is i; the bottom left corner is i2, and its pixel coordinates are (x, y). i2 y i2 The top right vertex is i3, corresponding to the bottom left corner of the rectangular road sign; its pixel coordinates are (x, y). i3 y i3 The top right corner of the rectangular road sign is i; the bottom right vertex is i4, and its pixel coordinates are (x, y). i4 y i4 The lower right corner of the rectangular road sign corresponds to this. The second marking area corresponding to the rectangular road sign is also a quadrilateral area, with its upper left vertex being i1' and its pixel coordinates being (x...). i1 ',y i1 '), corresponding to the top left corner of the rectangular road sign; the bottom left vertex is i2', and its pixel coordinates are (x, y). i2 ',y i2 The top right vertex is i3', corresponding to the bottom left corner of the rectangular road sign; its pixel coordinates are (x, y). i3 ',y i3 '), corresponding to the top right corner of the rectangular road sign; the bottom right vertex is i4', and its pixel coordinates are (x i4 ',y i4 The ') corresponds to the bottom right corner of the rectangular road sign. During matching, i1 is matched with i1', i2 with i2', i3 with i3', and i4 with i4'.

[0120] respectively (x i1 y i1 ) and (x i1 ′,y i1 ′), (x i2 y i2 ) and (x i2 ′,y i2 '), (x i3 y i3 ) and (x i3 ',y i3 '), (x i4 y i4 ) and (x i4 ',y i4 Substituting these values ​​into Equation 4, we obtain the correction matrices ΔT1, ΔT2, ΔT3, and ΔT4. We then select ΔT1, ΔT2, ΔT3, and ΔT4 to calculate the loss values, with ΔT1 having the smallest loss value.

[0121] Connect ΔT1 with (x) i1 y i1 Substituting into Equation 3, we obtain the pixel coordinates (x, y) of the reference point i1". i1 ” y i1 ”), and connect ΔT1 with (x i2 y i2 Substituting into Equation 3, we obtain the pixel coordinates (x, y) of reference point i2". i 2”y i2 ”), and connect ΔT1 with (x i3 y i3 Substituting into Equation 3, we obtain the pixel coordinates (x, y) of the reference point i3". i3 ” y i3 ”), and connect ΔT1 with (x i4 y i4 Substituting into Equation 3, we obtain the pixel coordinates (x, y) of reference point i4". i4 ” y i4 Set a distance threshold L, and calculate the distances I1 between i1” and i1’, I2 between i2” and i2’, I3 between i3” and i3’, and I4 between i4” and i4’ respectively. Compare I1, I2, I3, and I4 with L respectively. If I1, I2, and I3 are less than L, keep the points i1’, i2’, and i3’ in the second identification area corresponding to i1”, i2”, and i3”, and delete i4’.

[0122] Rematch the points i1', i2', and i3' in the second identification region with the points in the first identification region, i.e., match i1 with i1', i2 with i2', and i3 with i3'. Then repeat step S1352, substituting the pixel coordinates of each pair of matched points into Equation 4 to solve for the correction matrices ΔT1', ΔT2', and ΔT3'. Select ΔT1', ΔT2', and ΔT3' to calculate the loss value, where ΔT2' has the smallest loss value, and ΔT2' is the better correction matrix.

[0123] In some practical applications, the first marker region, confirmed by the user, serves as the basis for image correction. Therefore, the pixel coordinates of the first marker region are accurate. The second marker region, however, is obtained through the YOLOv5 image recognition algorithm. This second region may be inaccurate due to limitations in the algorithm's accuracy or image quality issues (such as poor lighting conditions during image acquisition or the marker being obscured by other objects). Some points may not be within the second marker region. When matching points in the first and second marker regions, some matching relationships will be incorrect. These errors will lead to deviations in the calculated optimal solutions for Δp ​​and Δt, resulting in lower accuracy of the correction matrix. To solve the above problem, the pixel coordinates of the reference point are obtained by substituting the correction matrix and the pixel coordinates of the points in the first identification region back into Equation 3. The reference point is then compared with the corresponding point in the second identification region. If the distance between the two points is greater than the preset distance threshold, it can be determined that the matching relationship between the point in the second identification region and the point in the corresponding first identification region is incorrect. After removing the point in the second identification region, the remaining points in the second identification region are matched with the points in the first identification region again. Repeating the above calculation steps will yield a better correction matrix.

[0124] S14. The pixel coordinates of the first region of interest are transformed by the correction matrix to obtain the first pixel coordinates.

[0125] Specifically, the pixel coordinates of the first region of interest are denoted as (x... j y j ), will (x j y j Substituting the first pixel coordinate (x') and the correction matrix into Equation 3, we obtain the first pixel coordinate (x') j y' j ):

[0126]

[0127] S15. Generate a bounding box based on the coordinates of the first pixel, use it as the second region of interest, and replace the first region of interest with the second region of interest to complete the correction.

[0128] The first and second identifier regions are regions formed on the first and second images respectively by the same identifier. Therefore, the pixel coordinates of the two regions can be converted to each other through matrix transformation. Similarly, the pixel coordinates of the first region of interest (ROI) are transformed using the aforementioned matrix transformation method, and a bounding box is generated based on the transformed pixel coordinates. The image within this bounding box reflects the target area in the monitoring scene, just as the image within the first ROI region does. This application calculates the correction matrix by combining the pixel coordinates of the first and second identifier regions with the imaging formula of the image acquisition device; then, the first pixel coordinates are obtained by combining the correction matrix with the pixel coordinates of the first ROI region; finally, a second ROI is generated on the second image based on the first pixel coordinates. The image within the second ROI region is the target area image in the monitoring scene, which is used to replace the first ROI region, thus solving the ROI offset problem.

[0129] When image acquisition equipment is used for illegal parking detection, the illegal parking area is set as ROI(. As the image acquisition equipment shifts, the ROI also shifts. The ROI(. after the shift) no longer reflects the situation in the illegal parking area. If it is not corrected, the detection system will record the vehicles in the shifted ROI as illegal vehicles, thus causing a misjudgment.

[0130] In a second aspect of the embodiment, this application provides a method for detecting illegal parking, specifically including the following steps:

[0131] S21. Identify the second region of interest using any of the aforementioned image correction methods, and use it as the new region of interest.

[0132] In some embodiments, the aforementioned illegal parking detection method further includes, before S21:

[0133] The triggering conditions for the image correction method are set, including at least one of the following conditions: a preset time interval, the image acquisition device moving to a preset position, and the receipt of a correction command. When any one of the triggering conditions is met, the image is corrected.

[0134] Image acquisition equipment misalignment is generally caused by aging of its own structure or changes in the installation environment. Therefore, the frequency of misalignment is not very high, and frequent image correction is unnecessary. By setting trigger conditions for image correction according to actual needs, image correction can be performed only when the trigger conditions are met, saving computing power.

[0135] S22. Recognize the second image, obtain vehicle information within the region of interest, record it as the first vehicle information, and configure a counter.

[0136] In some specific implementations, vehicle information in images is identified using image recognition algorithms. First, the YOLOv5 recognition algorithm is trained: The video images are manually labeled, categorizing vehicles into four types: cars, vans, trucks, and buses. After labeling, the YOLOv5 algorithm is used to detect the labeled vehicles, training the model to obtain the optimal detection model. Then, the YOLOv5 algorithm is used to identify vehicles in a second image, finding vehicle information within the second region of interest.

[0137] S23. Obtain the third image, which is the next frame of the second image. Recognize the third image to obtain vehicle information within the region of interest, which is recorded as the second vehicle information.

[0138] S24. Compare the second vehicle information with the first vehicle information. If they are the same, increment the counter by one; if they are different, delete the first vehicle information, record the second vehicle information, and start counting again.

[0139] Specifically, the aforementioned vehicle information refers to the outline region of the vehicle in the image. The overlap between the first and second vehicle information is calculated to determine if the vehicle information is identical. Let the first vehicle information be represented as M, with its area denoted as m, and the second vehicle information as N, with its area denoted as n. The overlapping region of M and N can be represented as M∩N, with its area denoted as I. When m≥n, the overlap is I / n; when m<n, the overlap is I / m; and when the overlap is greater than a preset value, the vehicle information is considered identical.

[0140] S25. By analogy, continue to acquire subsequent consecutive frame images, set a count threshold, and when the value of the counter exceeds the count threshold, determine that the vehicle has committed illegal parking behavior.

[0141] This method uses the aforementioned image correction method to correct the offset ROI during the illegal parking detection process, thus solving the problem of misjudgment of illegal parking caused by the offset of the image acquisition device.

[0142] In a third aspect of the embodiment, this application provides an image correction device 10, including a receiving module 11, an identification module 12, a first data processing module 13, a second data processing module 14, and a generation module 15.

[0143] The receiving module 11 is used to obtain the pixel coordinates of the first identified region and the pixel coordinates of the first region of interest.

[0144] The recognition module 12 is used to recognize the second image and obtain the pixel coordinates of the second identification area.

[0145] The first data processing module 13 is used to calculate the correction matrix and implement the processing steps S131 to S134 or S131 to S135, which will not be described in detail here.

[0146] The second data processing module 14 is used to transform the pixel coordinates of the first region of interest using a correction matrix to obtain the first pixel coordinates.

[0147] The generation module 15 is used to generate a bounding box on the second image based on the first pixel coordinates, and to use the area within the bounding box as the second region of interest, replacing the first region of interest.

[0148] It is understood that the various modules in the image correction device 10 can be integrated on a processor, or each module can exist separately in a different processor, or two or more modules can be integrated into a processor. In addition to being implemented in hardware such as processors, the above modules can also be implemented in the form of software functional modules.

[0149] Each unit module of the image correction device 10 can execute the corresponding steps in any of the above-mentioned marker recognition methods. Therefore, the details of each unit module will not be elaborated here. Please refer to the corresponding step description above for details.

[0150] In a fourth aspect of the embodiment, this application provides an illegal parking detection device 20, including the aforementioned image correction device 10, a first vehicle detection module 21, a second vehicle detection module 22, a comparison module 23, and a judgment module 24.

[0151] The image correction device 10 is used to perform image correction.

[0152] The first vehicle detection module 21 is used to identify the second image and obtain vehicle information within the region of interest, which is denoted as the first vehicle information.

[0153] The second vehicle detection module 22 is used to acquire a third image, which is the next frame of the second image. The third image is identified to obtain vehicle information in the region of interest, which is recorded as the second vehicle information.

[0154] The comparison module 23 is used to compare the second vehicle information with the first vehicle information. If they are the same, the consecutive frames are incremented by one; if they are different, the first vehicle information is deleted and the second vehicle information is recorded.

[0155] The judgment module 24 is used to determine whether the vehicle has violated regulations. When the number of consecutive frames exceeds a preset value, it is determined that the vehicle has violated parking regulations.

[0156] It is understandable that the various modules in the illegal parking detection device 20 can be integrated on one processor, or each module can exist separately in different processors, or two or more modules can be integrated into one processor. In addition to being implemented in hardware such as processors, the above modules can also be implemented in the form of software functional modules.

[0157] Each unit module of the illegal parking detection device 20 can execute the corresponding steps in any of the above-mentioned marker recognition methods. Therefore, the details of each unit module will not be elaborated here. Please refer to the corresponding step descriptions above for details.

[0158] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. An image correction method, characterized in that, Based on a first image and a second image acquired sequentially at the same location by the same image acquisition device, the first image is configured with a first region of interest and also includes a first identifier region, and the second image includes a second identifier region. The first identifier region and the second identifier region are regions formed by the same identifier on the first image and the second image respectively. The image correction method includes: Obtain the pixel coordinates of the first identification region and the second identification region, compare the pixel coordinates of the first identification region and the pixel coordinates of the second identification region, and obtain the correction matrix; Obtain the pixel coordinates of the first region of interest, and transform the pixel coordinates of the first region of interest through the correction matrix to obtain the second region of interest; The second region of interest replaces the first region of interest, thus completing the correction.

2. The image correction method according to claim 1, characterized in that, The correction matrix is: ， in, The angle of change of the pitch angle of the image acquisition device when acquiring the first image and the second image; The angle of change of the heading angle of the image acquisition device when acquiring the first image and the second image.

3. The image correction method according to claim 2, characterized in that, The correction matrix is ​​calculated in the following manner: The imaging formula of the image acquisition device is: ， Where (X, Y, Z) are world coordinates; (x, y) are pixel coordinates; This is the intrinsic parameter matrix; Here, t is the rotation matrix, p is the pitch angle, and p is the yaw angle. Let K denote the intrinsic parameter matrix and T denote the rotation matrix. Transforming the imaging formula, we obtain the reconstruction formula: ， The world coordinates of the identifier are set as follows: The pixel coordinates of the first identified region are defined as follows: The pixel coordinates of the second identified region are defined as follows: The rotation matrix used to acquire the first image is denoted as... The rotation matrix used to acquire the second image is denoted as... ;Will , and Substituting into the reduction formula, we get: ， Will , and Substituting into the imaging formula, we get: ， Substituting the reduction formula into the imaging formula yields the target formula: ， in, , Construct the objective function based on the stated objective formula: ， Solve using nonlinear optimization algorithms and The optimal solution is obtained, and then the correction matrix is ​​calculated.

4. The image correction method according to claim 3, characterized in that, The pixel coordinates of the first region of interest are transformed using the correction matrix to obtain the first pixel coordinates, and the second region of interest is generated based on the first pixel coordinates; the first pixel coordinates are calculated in the following way: The pixel coordinates of the first region of interest are denoted as ,Will The first pixel coordinates are obtained by substituting the correction matrix into the target formula. : 。 5. The image correction method according to claim 3, characterized in that, When both the first and second identification regions contain multiple points, the correction matrix is ​​obtained through the following steps: S1351. Match the points in the first identification area with the points in the second identification area to obtain multiple pairs of matching points; S1352. Substitute the pixel coordinates of the multiple pairs of matching points into the objective function to solve for multiple correction matrices, and select the correction matrix with the smallest loss value from the multiple correction matrices. S1353. Substitute the pixel coordinates of the points in the first identification region among the multiple pairs of matched points and the correction matrix with the smallest loss value into the target formula to obtain multiple reference points; S1354. Calculate the distance between each reference point and the corresponding point in the second identification area, select the reference points that are less than the distance threshold, and retain the points in the second identification area corresponding to the reference points that are less than the distance threshold. S1355. Rematch the points retained in the second identification area with the points in the first identification area, and repeat step S1352 to obtain a better correction matrix.

6. The image correction method according to claim 1, characterized in that, The signage includes: directional signs, signs, traffic markings, ground markings, parking lines, and shorelines.

7. A method for detecting illegal parking, characterized in that, include: The second region of interest is determined by the image correction method as described in any one of claims 1 to 6, and is used as a new region of interest. The second image is identified to obtain vehicle information within the region of interest, which is denoted as the first vehicle information. Acquire a third image, which is the next frame of the second image; identify the third image to obtain vehicle information within the region of interest, and record it as the second vehicle information. The second vehicle information is compared with the first vehicle information. If they are the same, the consecutive frames are incremented by one. If they are different, delete the first vehicle information and record the second vehicle information; This process continues, acquiring subsequent consecutive frames. When the number of consecutive frames exceeds a preset value, it is determined that the vehicle has engaged in illegal parking.

8. The method for detecting illegal parking according to claim 7, characterized in that, The illegal parking detection method further includes: setting the triggering conditions for the image correction method; the triggering conditions include at least one of the following: a preset time interval, the image acquisition device moving to a preset position, and receiving a correction command; when any of the triggering conditions is met, image correction is performed.

9. An image correction device, characterized in that, Based on a first image and a second image acquired sequentially at the same location by the same image acquisition device, a first region of interest is set on the first image, and the first image also includes a first identifier region, and the second image includes a second identifier region. The first identifier region and the second identifier region are regions formed by the same identifier on the first image and the second image respectively. The image correction device includes: The receiving module is used to obtain the pixel coordinates of the first identified region and the pixel coordinates of the first region of interest; The recognition module is used to recognize the second image and obtain the pixel coordinates of the second identified region; The first data processing module is used to calculate the correction matrix: The second data processing module is used to transform the pixel coordinates of the first region of interest through the correction matrix to obtain the first pixel coordinates; The generation module is used to generate a second region of interest on the second image based on the first pixel coordinates, and replace the first region of interest with it.

10. A device for detecting illegal parking, characterized in that, include: The image correction device as described in claim 9; The first vehicle detection module is used to identify the second image and obtain vehicle information within the second region of interest, which is denoted as the first vehicle information. The second vehicle detection module is used to acquire a third image, which is the next frame of the second image, and to identify the third image to obtain vehicle information within the region of interest, which is denoted as the second vehicle information. The comparison module is used to compare the second vehicle information with the first vehicle information. If they are the same, the consecutive frames are incremented by one. If they are different, delete the first vehicle information and record the second vehicle information; The judgment module is used to determine whether the vehicle has committed a traffic violation. When the number of consecutive frames exceeds a preset value, it is determined that the vehicle has committed an illegal parking violation.

Images