A single image-based berth line offset determination method and system

Abstract

The application discloses a berth line offset determination method and system based on monocular images, and relates to the field of intelligent parking management.The method comprises the following steps: obtaining data with a potential berth offset greater than or equal to a preset threshold based on the content recognition result of a single picture; combining an edge and straight line detection algorithm in image processing; calculating a berth line segment matching parameter sequence according to the line segment parameter set and the line segment intersection cluster information in the picture extracted from the data with an offset greater than or equal to the preset threshold to recognize the berth offset labeling situation.The algorithm can not only discover the berth offset situation in time, but also effectively overcome the instability of the current ground segmentation algorithm result, and improve the accuracy of the judgment result.The offset measurement data provided can be used to process the priority order of the offset target to be corrected, so as to reduce the system calculation burden, improve the overall working efficiency and the overall data automatic processing level, and further improve the accuracy and efficiency of the berth line recognition.

Description

Technical Field

[0001] This invention relates to the field of intelligent parking management, and in particular to a method and system for determining parking space line offset based on monocular images. Background Technology

[0002] Automated roadside parking fee collection systems have become deeply integrated into efficient urban operations management. For complex systems, the ability to quickly detect and promptly correct data anomalies is crucial for stable operation. In roadside high-position video parking monitoring, parking space line markings are the most important data for determining whether a vehicle occupies a parking space. However, due to uncertain external environmental factors, camera angles can change randomly due to external forces such as wind, causing significant deviations between the parking space line marking coordinates and the actual image position of the parking space. In such cases, it is necessary to correct the coordinates of the parking space vertex image to ensure the validity of subsequent data used by the algorithm and the accuracy of a series of algorithmic results. Therefore, quickly identifying deviations in parking space line marking coordinates is a critical step in the automated monitoring process.

[0003] Currently, in general, determining whether berth line markings have shifted requires first extracting features from the berth markings in the image and then comparing them with the marking data. However, in real-world applications, complex situations arise such as irregular, worn, obstructed, or curved ground markings. This causes simple recognition methods to be susceptible to numerous errors, resulting in extraction failures. For example, using edge and line detection in traditional image processing to extract line segment features is highly susceptible to abnormal interference, leading to low accuracy and efficiency in determining existing berth lines. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a method and system for determining berth line offset based on monocular images, which can solve the problems of low accuracy and efficiency in existing berth line determination methods.

[0005] To achieve the above objectives, on the one hand, the present invention provides a method for determining berth line offset based on monocular images. The method includes: extracting key pattern regions from the ground line segmentation map results based on the berth vertex coordinates, and filtering out data whose segmentation result offset is greater than or equal to a preset threshold.

[0006] Extract line segment parameter sets and line segment intersection cluster information from data with offsets greater than or equal to a preset threshold;

[0007] Based on the berth vertex data and the intersection cluster information of line segments in the image, obtain the matching parameter sequence of the four marked berth lines;

[0008] Based on the matching parameter sequence, determine whether the berth location data has shifted, and output the correct berth area vertex information and shift degree evaluation parameters.

[0009] Furthermore, the step of extracting key pattern regions from the ground line segmentation map results based on the berth vertex coordinates and filtering out data whose segmentation result offset is greater than or equal to a preset threshold includes:

[0010] Calculate the virtual world coordinate system point group corresponding to the order of the berth vertices based on the physical size information of the standard berth, and calculate the inverse perspective transformation matrix and the corresponding inverse matrix of the berth vertices based on the paired point group;

[0011] In the virtual world coordinate system, the physical dimensions of the berth are scaled by a preset ratio to obtain the coordinates of the inner and outer frames of the berth line. Based on the perspective transformation matrix, the corresponding coordinates of the inner and outer frames of the berth line in the image coordinate system, the corresponding polygon area, and the area of ​​the annular region in the form of two nested polygons are calculated.

[0012] Based on the corresponding coordinates of the inner and outer frame coordinates of the berth line in the image coordinate system, obtain the annular region sub-image of the segmentation image corresponding to the nested form of the berth line;

[0013] The consistency parameter between the segmented sub-map and the berth labeling pattern is calculated based on the area of ​​the annular region in the two nested polygon forms, and data in the consistency parameter between the segmented sub-map and the berth labeling pattern that is greater than or equal to a preset threshold is obtained.

[0014] Furthermore, the step of extracting the line segment parameter set and the line segment intersection cluster information within the image from data with an offset greater than or equal to a preset threshold includes:

[0015] Preprocess the ground line segmentation map corresponding to the data whose offset is greater than or equal to a preset threshold to obtain the endpoint parameters of the image line segments;

[0016] Based on the endpoint parameters of the line segments in the image, the line segments in the adjacent area with approximate direction, near-line perpendicular distance, and near-center point distance are merged, and the line segment feature parameters are updated;

[0017] The updated line segment feature parameters are then merged a second time to obtain line segment clusters;

[0018] For each of the merged line segment clusters obtained above, calculate the coordinates of the intersection points of the pairs of lines, and save the intersection point coordinates and the corresponding intersecting line segment identifiers within the image to generate line segment intersection point cluster information within the image.

[0019] Further, the step of obtaining the matching parameter sequence of the four marked berth lines based on the berth vertex data and the intersection cluster information of line segments in the image includes:

[0020] Match the coordinates of the vertices of the four berth edges with the straight lines in the intersection cluster information of the line segments in the image to obtain the optimal candidate detection straight line of the berth line, and record the straight line distance and angle between the marked berth line and the candidate detection line.

[0021] The system determines whether the marked berth line matches the optimal candidate detection line based on the straight-line distance and the included angle between the marked berth line and the candidate detection line.

[0022] If there is no match, the matching parameter sequence is obtained based on the straight-line distance and angle between the marked berth line and the candidate detection line.

[0023] Further, the step of determining whether the berth positioning data has shifted based on the matching parameter sequence, and outputting the correct berth area vertex information and shift degree evaluation parameters includes:

[0024] Based on the berth line matching parameter sequence and scene segmentation line features, configure the berth line offset type corresponding to different sequence modes;

[0025] By matching the detection line identifier sequence of each berth marking line and the information of the intersection point cluster of line segments in the image, the vertex information of the correct berth area is obtained;

[0026] The comprehensive offset parameter sequence of the berth is calculated by matching the detection line offset measurement information sequence of the marking lines of each berth.

[0027] On the other hand, the present invention provides a berth line offset determination system based on monocular images. The system includes: an extraction unit, used to extract key pattern regions from the ground line segmentation map results based on the berth vertex coordinates, and filter out data whose segmentation result offset is greater than or equal to a preset threshold.

[0028] The extraction unit is also used to extract line segment parameter sets and line segment intersection cluster information in the image from data with an offset greater than or equal to a preset threshold.

[0029] The acquisition unit is used to acquire the matching parameter sequence of four marked berth lines based on the berth vertex data and the line segment intersection cluster information in the image;

[0030] The judgment output unit is used to determine whether the berth positioning data has shifted based on the matching parameter sequence, and output the correct berth area vertex information and the degree of shift evaluation parameters.

[0031] Further, the extraction unit is specifically used to calculate the virtual world coordinate system point group corresponding to the order of the berth vertices based on the physical size information of the standard berth, and to calculate the inverse perspective transformation matrix and the corresponding inverse matrix of the berth vertices based on the paired point group; to scale the physical size of the berths according to a preset ratio in the virtual world coordinate system to obtain the coordinates of the inner and outer frames of the berth lines, and to calculate the corresponding coordinates of the inner and outer frames of the berth lines in the image coordinate system, the corresponding polygon area, and the area of ​​the annular region in the form of two nested polygons based on the perspective transformation matrix; to obtain the annular region sub-image of the segmented image corresponding to the nested berth lines based on the corresponding coordinates of the inner and outer frames of the berth lines in the image coordinate system; to calculate the consistency parameter between the segmented sub-image and the berth labeling mode based on the area of ​​the annular region in the form of two nested polygons, and to obtain the data in the consistency parameter between the segmented sub-image and the berth labeling mode that is greater than or equal to a preset threshold.

[0032] Furthermore, the extraction unit is specifically used to preprocess the ground line segmentation map corresponding to the data with an offset greater than or equal to a preset threshold to obtain the endpoint parameters of the image line segments; merge the line segments with approximate direction, near-line perpendicular distance, and near-center point distance in the adjacent area according to the endpoint parameters of the image line segments, and update the line segment feature parameters; perform a second merging on the updated line segment feature parameters to obtain line segment clusters; calculate the coordinates of the intersection points of each pair of lines in the merged line segment clusters obtained above, and save the intersection point coordinate information and the corresponding intersecting line segment identification information located in the image to generate line segment intersection point cluster information in the image.

[0033] Furthermore, the acquisition unit is specifically used to match the coordinates of the vertices of the four berth edges with the straight lines in the intersection cluster information of the line segments in the image, to obtain the optimal candidate detection straight line of the berth line, and to record the straight line distance and angle between the marked berth line and the candidate detection line; and to determine whether the marked berth line and the optimal candidate detection line match based on the straight line distance and angle between the marked berth line and the candidate detection line; if they do not match, then the matching parameter sequence is obtained based on the straight line distance and angle between the marked berth line and the candidate detection line.

[0034] Furthermore, the judgment output unit is specifically used to configure the berth line offset type corresponding to different sequence modes based on the berth line matching parameter sequence and scene segmentation line features; obtain the vertex information of the correct berth area through the detection line identifier sequence of each berth label line matching and the line segment intersection cluster information in the image; and calculate the comprehensive offset parameter sequence of the berth through the detection line offset metric information sequence of each berth label line matching.

[0035] This invention provides a method and system for determining berth line offset based on monocular images. It extracts line segment parameter sets and line segment intersection cluster information from data with offsets greater than or equal to a preset threshold based on the content recognition results of a single image. Combining edge and line detection algorithms in image processing, it calculates a berth line segment matching parameter sequence based on the extracted line segment parameter sets and the line segment intersection cluster information to identify berth offsets. This not only allows for timely detection of berth offsets but also effectively overcomes the instability of current algorithm results, improving the accuracy of the judgment. Furthermore, the provided offset measurement data can prioritize the processing of offset targets to be corrected, reducing the system's computational burden and improving overall work efficiency and the level of automated data processing, thereby enhancing the accuracy and efficiency of berth line recognition. Attached Figure Description

[0036] Figure 1 This is a flowchart of a method for determining berth line offset based on monocular images provided by the present invention;

[0037] Figure 2 This is a schematic diagram of the structure of a berth line offset determination system based on monocular images provided by the present invention. Detailed Implementation

[0038] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0039] like Figure 1 As shown in the figure, an embodiment of the present invention provides a method for determining berth line offset based on monocular images, which includes the following steps:

[0040] 101. Based on the berth vertex coordinates, extract key pattern regions from the ground line segmentation map results and filter out data whose segmentation result offset is greater than or equal to a preset threshold.

[0041] In this embodiment of the invention, step 101 may specifically include: calculating a virtual world coordinate system point group corresponding to the order of the berth vertices based on the physical size information of the standard berth, and calculating the inverse perspective transformation matrix and the corresponding inverse matrix of the berth vertices based on the paired point group; scaling the physical size of the berths according to a preset ratio in the virtual world coordinate system to obtain the coordinates of the inner and outer frames of the berth lines, and calculating the corresponding coordinates of the inner and outer frames of the berth lines in the image coordinate system, the corresponding polygon area, and the area of ​​the annular region in the form of two nested polygons based on the perspective transformation matrix; obtaining the annular region sub-image of the segmented image corresponding to the nested berth lines based on the corresponding coordinates of the inner and outer frames of the berth lines in the image coordinate system; calculating the consistency parameter between the segmented sub-image and the berth labeling mode based on the area of ​​the annular region in the form of two nested polygons, and obtaining data in the consistency parameter between the segmented sub-image and the berth labeling mode that is greater than or equal to a preset threshold.

[0042] For example, step 101.1: Assume a set of physical dimensions (w, h) of the berths according to the general berth size ratio, calculate the virtual world coordinate system point group berth_ann_wrd corresponding to the order of the berth_ann vertices of the berths labeled in the image, and calculate the inverse perspective transformation matrix Mp and its inverse matrix Mp_inv of the image coordinate points based on the paired point groups; Step 101.2: Extract the key pattern segmentation region by extending the line width of the labeled berth lines, which is specifically divided into the following two steps:

[0043] In the virtual world coordinate system, the physical dimensions of the berth are scaled by a certain percentage (e.g., 15%) to obtain the coordinates of the inner and outer frames of the berth line: berth_ann_in_wrd\berth_ann_out_wrd. The perspective transformation matrix Mp_inv is used to calculate the corresponding coordinates berth_ann_in\berth_ann_out in the image coordinate system, as well as the corresponding polygon areas Area_in\Area_out. Furthermore, the area of ​​the two nested polygon-shaped annular regions, Dd_area = Area_out - Area_in, is calculated. ; Use the coordinate point clusters berth_ann_in and berth_ann_out to obtain the ring-shaped sub-map Smask corresponding to the nested berth lines of the segmentation map Segmap, and calculate the consistency parameter segrat = sum(Smask) / Dd_area / 255 between the segmentation sub-map Smask and the berth labeling mode; Set a threshold thr. If segrat >= thr, it can be determined that the berth has not been offset, and the offset judgment index if_shift = -1 is set. Otherwise, further subsequent judgment is required, and the offset judgment index if_shift = 0 is set.

[0044] 102. Extract line segment parameter sets and line segment intersection cluster information from data with offsets greater than or equal to a preset threshold.

[0045] In this embodiment of the invention, step 102 may specifically include: preprocessing the ground line segmentation map corresponding to the data with an offset greater than or equal to a preset threshold to obtain the endpoint parameters of the image line segments; merging line segments in the adjacent area with approximate direction, near-line perpendicular distance, and near-center point distance according to the endpoint parameters of the image line segments, and updating the line segment feature parameters; performing a second merging on the updated line segment feature parameters to obtain line segment clusters; calculating the coordinates of the intersection points of each pair of lines in each of the merged line segment clusters obtained above, and saving the intersection point coordinate information and the corresponding intersecting line segment identification information located in the image to generate line segment intersection point cluster information in the image.

[0046] For example, step 102.1: For the ground line segmentation map Segmap, obtain preliminary line detection results through basic image processing methods: such as Gaussian denoising, Canny edge detection, and Hough line detection to obtain the endpoint parameters of the image line segments; Step 102.2: For the line segment cluster A0 obtained above, merge the line segments with approximate directions, near-line perpendicular distances, and near-center distances in the neighboring regions, and update the line segment feature parameters, such as slope, intercept, length, endpoint information, etc., to form a new line segment cluster A1. Step 102.3: Perform a second merging on the line segment cluster A1 obtained above to obtain a new line segment cluster A2: create a point feature vector composed of tilt angle and position coordinate information, and automatically calculate the number of clusters based on the statistical distribution data of the feature vector. Then, use a clustering algorithm to divide the targets in line segment cluster A1 into clusters using the previously formed point feature vectors, and perform point fitting within each cluster. If the fitting quality meets the conditions, update the new line segment parameters; otherwise, retain the original line segment parameters, and finally form a new line segment cluster A2. Step 102.4: For the merged line segment cluster A2 obtained above, calculate the coordinates of the intersection points of each pair of lines, and retain the intersection point coordinates and the corresponding intersecting line segment IDs in the image to form the data itsp_list.

[0047] 103. Based on the berth vertex data and the line segment intersection cluster information in the image, obtain the matching parameter sequence for the four marked berth lines.

[0048] In this embodiment of the invention, step 103 may specifically include: matching the coordinates of the vertices of the four berth edges with the straight lines in the intersection cluster information of the line segments in the image to obtain the optimal candidate detection straight line of the berth line, and recording the straight line distance and angle between the marked berth line and the candidate detection line; and determining whether the marked berth line and the optimal candidate detection line match based on the straight line distance and angle between the marked berth line and the candidate detection line; if they do not match, obtaining the matching parameter sequence based on the straight line distance and angle between the marked berth line and the candidate detection line.

[0049] For example, the coordinates of the four berth edge vertices are obtained based on the berth label vertex information bth_ann, and each vertex is matched with the straight line in the straight line cluster information A2 to obtain the optimal candidate detection line L_idx[i] = id (i = 1, 2, 3, 4). The straight line distance L_dist[i] = dist and the included angle L_paral[i] = thr_dist between the labeled berth line and the candidate detection line are recorded. Based on these two line segment deviation measures, it is determined whether the labeled berth line matches the optimal candidate detection line. If there is a candidate line segment and it matches, then L_match[i] = 1. If there is a candidate line segment but it does not match, then L_match[i] = -1. If there is no candidate line segment, then L_match[i] = 0. In this way, four berth line matching result sequences of length 4 are obtained: L_match, L_idx, L_dist, and L_paral. The method for calculating the optimal candidate detection line is as follows: The coordinates of the two endpoints L_ann = [x1, y1, x2, y2] of each marked berth line are obtained sequentially, and the line distance dist and the line angle thr_dist between each detection line and L_det_j = [xx1, yy1, xx2, yy2] are calculated. These are then combined into a deviation parameter score = f(dist) + g(thr_dist) through normalization. The detection line id with the smallest score is selected as the optimal candidate detection line.

[0050] 104. Determine whether the berth positioning data has shifted based on the matching parameter sequence, and output the correct berth area vertex information and the degree of shift evaluation parameters.

[0051] In this embodiment of the invention, step 104 may specifically include: configuring berth line offset types corresponding to different sequence modes based on the berth line matching parameter sequence and scene segmentation line features; obtaining the vertex information of the correct berth area through the detection line identifier sequence of each berth marker line matching and the line segment intersection cluster information in the image; and calculating the comprehensive offset parameter sequence of the berth through the detection line offset metric information sequence of each berth marker line matching.

[0052] For example, based on the berth line matching parameter sequence L_match and scene segmentation features obtained above, the berth line offset situation corresponding to different sequence modes is set. If it is judged that the berth line has not been offset, the offset judgment index if_shift = -1 is set; if it is judged that the berth line label has been offset, the offset judgment index if_shift = 1 is set; otherwise, it is considered that the current result cannot be judged, and the offset judgment index if_shift = 0 is set. At this time, it is necessary to re-extract the camera sampling image for judgment. Among them, the calculation method of berth offset judgment by berth line matching parameter sequence L_match is as follows: ① Calculate the number of matching line segments mid_len and the number of unmatched line segments nmid_len by taking the value of berth line matching parameter sequence L_match respectively, and obtain the number of valid matching line segments vid_len = mid_len + nmid_len, and the number of invalid matching line segments nvid_len = 4 - vid_len; A. Judge according to the matching line segment mode: if vid_len = 0, then if_shift = 0; if vid_len <= 2: when L_match[1] + L If _match[3]<0 or L_match=[-1,0,-1,0], if_shift=1; if vid_len=2 and L_match[1]+L_match[3]=1 and L_match[0]+L_match[2]=1, if_shift=-1; otherwise if_shift=0; if vid_len>=3: if L_match[1]+L_match[3]>=0 and L_match[0]+L_match[2]>=1, if_shift=-1; otherwise if_shift=1.

[0053] Then, using the detection line ID sequence L_idx matched with the marking lines of each berth and the calculated detection line intersection information itsp_list, the potential correct berth vertex coordinate point group candi_points is further filtered according to the line segment ID. Finally, using the detection line offset metric information sequence L_dist and L_paral matched with the marking lines of each berth, the comprehensive offset parameter sequence metric = [max(L_dist), max(L_paral), nvid_len] of the berth is calculated. The larger the first two parameters are, the greater the berth marking deviation. The larger the last parameter is, the higher the uncertainty of the result, which requires subsequent checks and judgments by other algorithms. These evaluation parameters can be used to prioritize the offset berths to be corrected.

[0054] This invention provides a method for determining berth line offset based on monocular images. It extracts line segment parameter sets and line segment intersection cluster information from data with offsets greater than or equal to a preset threshold based on the content recognition results of a single image. Combining edge and line detection algorithms in image processing, it calculates a berth line segment matching parameter sequence based on the extracted line segment parameter sets and the line segment intersection cluster information to identify and mark berth offsets. This method not only promptly detects berth offsets but also effectively overcomes the instability of current algorithm results, improving the accuracy of the judgment. Furthermore, the provided offset measurement data can prioritize the processing of offset targets to be corrected, reducing the system's computational burden and improving overall work efficiency and the level of automated data processing, thereby enhancing the accuracy and efficiency of berth line recognition.

[0055] To implement the method provided in the embodiments of the present invention, the embodiments of the present invention provide a berth line offset determination system based on monocular images, such as... Figure 2 As shown, the system includes: an extraction unit 21, an acquisition unit 22, and a judgment and output unit 23.

[0056] Extraction unit 21 is used to extract key mode areas from the ground line segmentation map results based on the berth vertex coordinates, and filter out data whose segmentation result offset is greater than or equal to a preset threshold.

[0057] The extraction unit 21 is also used to extract line segment parameter sets and line segment intersection cluster information in the image from data with an offset greater than or equal to a preset threshold.

[0058] The acquisition unit 22 is used to acquire the matching parameter sequence of four marked berth lines based on the berth vertex data and the line segment intersection cluster information in the image;

[0059] The judgment output unit 23 is used to determine whether the berth positioning data has shifted based on the matching parameter sequence, and output the correct berth area vertex information and the degree of shift evaluation parameters.

[0060] Further, the extraction unit 21 is specifically used to calculate the virtual world coordinate system point group corresponding to the order of the berth vertices based on the physical size information of the standard berth, and to calculate the inverse perspective transformation matrix and the corresponding inverse matrix of the berth vertices based on the paired point group; to scale the physical size of the berths according to a preset ratio in the virtual world coordinate system to obtain the coordinates of the inner and outer frames of the berth lines, and to calculate the corresponding coordinates of the inner and outer frame coordinates of the berth lines in the image coordinate system, the corresponding polygon area, and the area of ​​the annular region in the form of two nested polygons based on the perspective transformation matrix; to obtain the annular region sub-image of the segmented image corresponding to the nested berth lines based on the corresponding coordinates of the inner and outer frame coordinates of the berth lines in the image coordinate system; to calculate the consistency parameter between the segmented sub-image and the berth labeling mode based on the area of ​​the annular region in the form of two nested polygons, and to obtain the data in the consistency parameter between the segmented sub-image and the berth labeling mode that is greater than or equal to a preset threshold.

[0061] Furthermore, the extraction unit 21 is specifically used to preprocess the ground line segmentation map corresponding to the data with an offset greater than or equal to a preset threshold to obtain the endpoint parameters of the image line segments; merge the line segments with approximate direction, near-line perpendicular distance, and near-center point distance in the adjacent area according to the endpoint parameters of the image line segments, and update the line segment feature parameters; perform a second merging on the updated line segment feature parameters to obtain line segment clusters; calculate the coordinates of the intersection points of each pair of lines in the merged line segment clusters obtained above, and save the intersection point coordinate information and the corresponding intersecting line segment identification information located in the image to generate line segment intersection point cluster information in the image.

[0062] Further, the acquisition unit 22 is specifically used to match the coordinates of the vertices of the four berth edges with the straight lines in the intersection cluster information of the line segments in the image, to obtain the optimal candidate detection straight line of the berth line, and to record the straight line distance and angle between the marked berth line and the candidate detection line; and to determine whether the marked berth line and the optimal candidate detection line match based on the straight line distance and angle between the marked berth line and the candidate detection line; if they do not match, then the matching parameter sequence is obtained based on the straight line distance and angle between the marked berth line and the candidate detection line.

[0063] Furthermore, the judgment output unit 23 is specifically used to configure the berth line offset type corresponding to different sequence modes according to the berth line matching parameter sequence and scene segmentation line features; obtain the vertex information of the correct berth area through the detection line identifier sequence of each berth label line matching and the line segment intersection cluster information in the image; and calculate the comprehensive offset parameter sequence of the berth through the detection line offset metric information sequence of each berth label line matching.

[0064] This invention provides a berth line offset determination system based on monocular images. It extracts line segment parameter sets and line segment intersection cluster information from data with offsets greater than or equal to a preset threshold based on the content recognition results of a single image. Combining edge and line detection algorithms in image processing, it calculates a berth line segment matching parameter sequence based on the extracted line segment parameter sets and the line segment intersection cluster information to identify berth offsets. This system not only promptly detects berth offsets but also effectively overcomes the instability of current algorithm results, improving the accuracy of the judgment. Furthermore, the provided offset measurement data can prioritize the processing of offset targets to be corrected, reducing the system's computational burden and improving overall work efficiency and the level of automated data processing, thereby enhancing the accuracy and efficiency of berth line recognition.

[0065] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process may be rearranged without departing from the scope of this disclosure. The appended method claims provide elements of various steps in an exemplary order and are not intended to limit the scope to the specific order or hierarchy described.

[0066] In the above detailed description, various features are combined together in a single embodiment to simplify this disclosure. This approach to disclosure should not be construed as reflecting an intention that embodiments of the claimed subject matter require more features than are explicitly stated in each claim. Rather, as reflected in the appended claims, the invention is presented with fewer features than all of the features of the single disclosed embodiment. Therefore, the appended claims are hereby explicitly incorporated into the detailed description, wherein each claim stands alone as a preferred embodiment of the invention.

[0067] The disclosed embodiments have been described above to enable any person skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments without departing from the spirit and scope of this disclosure. Therefore, this disclosure is not limited to the embodiments given herein, but is consistent with the broadest scope of the principles and novel features disclosed in this application.

[0068] The foregoing description includes examples of one or more embodiments. It is certainly impossible to describe all possible combinations of components or methods in order to describe the above embodiments, but those skilled in the art will recognize that further combinations and arrangements of the various embodiments are possible. Therefore, the embodiments described herein are intended to cover all such changes, modifications, and variations that fall within the scope of the appended claims. Furthermore, the term "comprising" as used in the specification or claims is interpreted in a manner similar to the term "including," as interpreted when used as a conjunction in the claims. Additionally, the use of any term "or" in the specification of the claims is intended to mean "non-exclusive or."

[0069] Those skilled in the art will also understand that the various illustrative logical blocks, units, and steps listed in the embodiments of the present invention can be implemented by electronic hardware, computer software, or a combination of both. To clearly demonstrate the interchangeability of hardware and software, the functions of the various illustrative components, units, and steps described above have been generally described. Whether such functionality is implemented through hardware or software depends on the specific application and the overall system design requirements. Those skilled in the art can implement the described functions using various methods for each specific application, but such implementation should not be construed as exceeding the scope of protection of the embodiments of the present invention.

[0070] The various illustrative logic blocks or units described in the embodiments of this invention can be implemented or operate the described functions using a general-purpose processor, digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof. The general-purpose processor can be a microprocessor; alternatively, it can be any conventional processor, controller, microcontroller, or state machine. The processor can also be implemented using a combination of computing devices, such as a digital signal processor and a microprocessor, multiple microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration.

[0071] The steps of the methods or algorithms described in the embodiments of this invention can be directly embedded in hardware, a software module executed by a processor, or a combination of both. The software module can be stored in RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from and write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and storage medium can be housed in an ASIC, which can be housed in a user terminal. Optionally, the processor and storage medium can also be housed in different components of the user terminal.

[0072] In one or more exemplary designs, the functions described in the embodiments of the present invention can be implemented in hardware, software, firmware, or any combination of these three. If implemented in software, these functions can be stored on a computer-readable medium or transmitted on a computer-readable medium in the form of one or more instructions or code. Computer-readable media include computer storage media and communication media that facilitate the transfer of computer programs from one place to another. Storage media can be any available media that can be accessed by a general-purpose or special-purpose computer. For example, such computer-readable media can include, but is not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store program code in the form of instructions or data structures and other forms that can be read by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Furthermore, any connection can be suitably defined as a computer-readable medium, for example, if the software is transmitted from a website, server or other remote resource via a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wirelessly, such as infrared, wireless and microwave, it is also included in the defined computer-readable medium. The disks and discs mentioned include compressed disks, laser discs, optical discs, DVDs, floppy disks, and Blu-ray discs. Disks typically copy data magnetically, while disks typically copy data optically using lasers. Combinations of the above can also be contained in computer-readable media.

[0073] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for determining berth line offset based on monocular images, characterized in that, The method includes: Based on the berth vertex coordinates, the key pattern regions of the ground line segmentation map are extracted, and data with segmentation offsets greater than or equal to a preset threshold are selected. Extract line segment parameter sets and line segment intersection cluster information from data with offsets greater than or equal to a preset threshold; The steps of extracting line segment parameter sets and line segment intersection cluster information from data with offsets greater than or equal to a preset threshold include: Preprocess the ground line segmentation map corresponding to the data whose offset is greater than or equal to a preset threshold to obtain the endpoint parameters of the image line segments; Based on the endpoint parameters of the line segments in the image, the line segments in the adjacent area with approximate direction, near-line perpendicular distance, and near-center point distance are merged, and the line segment feature parameters are updated; The updated line segment feature parameters are then merged a second time to obtain line segment clusters; For each of the merged line segment clusters obtained above, calculate the coordinates of the intersection points of the pairs of lines, and save the intersection point coordinates and the corresponding intersecting line segment identification information within the image to generate line segment intersection point cluster information within the image; Based on the berth vertex data and the intersection cluster information of line segments in the image, obtain the matching parameter sequence of the four marked berth lines; The step of obtaining the matching parameter sequence of the four marked berth lines based on the berth vertex data and the intersection point cluster information of line segments in the image includes: Match the coordinates of the vertices of the four berth edges with the straight lines in the intersection cluster information of the line segments in the image to obtain the optimal candidate detection straight line of the berth line, and record the straight line distance and angle between the marked berth line and the candidate detection line. The system determines whether the marked berth line matches the optimal candidate detection line based on the straight-line distance and the included angle between the marked berth line and the candidate detection line. If there is no match, the matching parameter sequence is obtained based on the straight-line distance and angle between the marked berth line and the candidate detection line; Based on the matching parameter sequence, determine whether the berth location data has shifted, and output the correct berth area vertex information and shift degree evaluation parameters.

2. The method for determining berth line offset based on monocular images according to claim 1, characterized in that, The step of extracting key pattern regions from the ground line segmentation map results based on the berth vertex coordinates and filtering out data whose segmentation result offset is greater than or equal to a preset threshold includes: Calculate the virtual world coordinate system point group corresponding to the order of the berth vertices based on the physical size information of the standard berth, and calculate the inverse perspective transformation matrix and the corresponding inverse matrix of the berth vertices based on the paired point group; In the virtual world coordinate system, the physical dimensions of the berth are scaled by a preset ratio to obtain the coordinates of the inner and outer frames of the berth line. Based on the inverse perspective transformation matrix, the corresponding coordinates of the inner and outer frames of the berth line in the image coordinate system, the corresponding polygon area, and the area of ​​the annular region in the form of two nested polygons are calculated. Based on the corresponding coordinates of the inner and outer frame coordinates of the berth line in the image coordinate system, obtain the annular region sub-image of the segmentation image corresponding to the nested form of the berth line; The consistency parameter between the segmented sub-map and the berth labeling pattern is calculated based on the area of ​​the annular region in the two nested polygon forms, and data in the consistency parameter between the segmented sub-map and the berth labeling pattern that is greater than or equal to a preset threshold is obtained.

3. The method for determining berth line offset based on monocular images according to claim 1, characterized in that, The step of determining whether the berth positioning data has shifted based on the matching parameter sequence, and outputting the correct berth area vertex information and shift degree evaluation parameters, includes: Based on the berth line matching parameter sequence and scene segmentation line features, configure the berth line offset type corresponding to different sequence modes; By matching the detection line identifier sequence of each berth marking line and the information of the intersection point cluster of line segments in the image, the vertex information of the correct berth area is obtained; The comprehensive offset parameter sequence of the berth is calculated by matching the detection line offset measurement information sequence of the marking lines of each berth.

4. A berth line offset determination system based on monocular images, characterized in that, The system includes: The extraction unit is used to extract key pattern areas from the ground line segmentation map results based on the berth vertex coordinates, and filter out data whose segmentation result offset is greater than or equal to a preset threshold. The extraction unit is also used to extract line segment parameter sets and line segment intersection cluster information in the image from data with an offset greater than or equal to a preset threshold. The extraction unit is further configured to preprocess the ground line segmentation map corresponding to the data with an offset greater than or equal to a preset threshold to obtain the endpoint parameters of the image line segments; merge the line segments with approximate direction, near-line perpendicular distance, and near-center point distance in the adjacent area according to the endpoint parameters of the image line segments, and update the line segment feature parameters; perform a second merging on the updated line segment feature parameters to obtain line segment clusters; calculate the coordinates of the intersection points of each pair of lines in the merged line segment clusters obtained above, and save the intersection point coordinate information and the corresponding intersecting line segment identification information located in the image to generate line segment intersection point cluster information in the image; The acquisition unit is used to acquire the matching parameter sequence of four marked berth lines based on the berth vertex data and the line segment intersection cluster information in the image; The acquisition unit is specifically used to match the coordinates of the vertices of the four berth edges with the straight lines in the intersection cluster information of the line segments in the image, to obtain the optimal candidate detection straight line of the berth line, and to record the straight line distance and angle between the marked berth line and the candidate detection line; and to determine whether the marked berth line and the optimal candidate detection line match based on the straight line distance and angle between the marked berth line and the candidate detection line; if they do not match, the matching parameter sequence is obtained based on the straight line distance and angle between the marked berth line and the candidate detection line. The judgment output unit is used to determine whether the berth positioning data has shifted based on the matching parameter sequence, and output the correct berth area vertex information and the degree of shift evaluation parameters.

5. A berth line offset determination system based on monocular images according to claim 4, characterized in that, The extraction unit is specifically used to calculate the virtual world coordinate system point group corresponding to the order of the berth vertices based on the physical size information of the standard berth, and to calculate the inverse perspective transformation matrix and the corresponding inverse matrix of the berth vertices based on the paired point group; to scale the physical size of the berths according to a preset ratio in the virtual world coordinate system, to obtain the coordinates of the inner and outer frames of the berth line, and to calculate the corresponding coordinates of the inner and outer frames of the berth line in the image coordinate system, the corresponding polygon area, and the area of ​​the annular region in the form of two nested polygons based on the inverse perspective transformation matrix; Based on the corresponding coordinates of the inner and outer frames of the berth lines in the image coordinate system, obtain the annular sub-image of the berth line nesting form in the segmentation image; calculate the consistency parameter between the segmentation sub-image and the berth labeling mode based on the area of ​​the annular region of the two polygon nesting forms, and obtain the data in the consistency parameter between the segmentation sub-image and the berth labeling mode that is greater than or equal to a preset threshold.

6. The berth line offset determination system based on monocular images according to claim 4, characterized in that, The judgment output unit is specifically used to configure the berth line offset type corresponding to different sequence modes based on the berth line matching parameter sequence and scene segmentation line features; obtain the vertex information of the correct berth area through the detection line identifier sequence of each berth label line matching and the line segment intersection cluster information in the image; and calculate the comprehensive offset parameter sequence of the berth through the detection line offset metric information sequence of each berth label line matching.

Images