Large model-based berth detection system and method

Abstract

The invention discloses a parking space detection system and method based on a large model, and relates to the technical field of intelligent transportation, and the system comprises an image recognition module which is used for recognizing a region where a parking space wireframe may exist and a region where a vehicle may exist in a snapshot image through a cross-modal comparative learning method; the image processing module is used for screening parking areas and vehicle areas according to the similarity among the multi-modal data and carrying out binarization processing and morphological analysis on the screened parking area images; the connected body identification module is used for carrying out connected body identification on the parking area image and extracting the attribute of a connected body; the centroid clustering module is used for calculating the centroid of the area image and clustering the centroid of the parking area image; and the position judgment module is used for judging the relative position of the berth and the vehicle according to the barycenter of the berth and the vehicle. According to the technical scheme of the invention, the method can achieve the recognition of the parking space and the vehicle, can judge the relative position of the parking space and the vehicle, and is higher in application value.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation technology, and in particular to a berth detection system and method based on a large model. Background Art

[0002] With the rapid development of urban economy and the continuous improvement of people's living standards, cars have become an indispensable means of transportation for people's daily travel, and the contradiction between parking needs has become increasingly acute. Especially on both sides of urban roads, due to the scarcity of roadside parking spaces, finding a suitable parking space has become a headache for many car owners. In recent years, some areas have adopted parking management methods based on high-position video and achieved good management results. However, since high-position video cameras are usually installed on one side of the road, the lens angle is relatively single, and coupled with the influence of factors such as tree occlusion and passing car occlusion, it may not be possible to accurately identify the relative position of the parking space and the vehicle.

[0003] With the development of technologies such as the Internet of Things, artificial intelligence, and big data, the rise of intelligent transportation technology has provided new possibilities for real-time monitoring and optimized management of parking spaces. Using computer vision, deep learning, and the Internet of Things, the intelligent transportation system can monitor the status of parking spaces in real time, improve the management efficiency of parking spaces, and optimize the allocation of parking resources. Therefore, on this basis, this application proposes a parking space detection system and method based on a large model. Summary of the invention

[0004] Technical Purpose In order to solve the above problems, the purpose of the present invention is to provide a parking space detection system and method based on a large model, which can not only realize the recognition function of on-road parking spaces and vehicles, but also can determine the relative position of parking spaces and vehicles. The method and system have a wide range of application scenarios, can realize automated and high-precision parking space detection, and the detection logic complexity is low.

[0005] Technical Solution In order to achieve the above-mentioned purpose, the present invention provides a parking space detection system and method based on a large model, which performs comparative learning of captured images and description texts through a CLIP model, maps the captured images and description texts into a common vector space, realizes cross-modal comparative learning, identifies parking space wireframes and vehicles in captured images through the CLIP model and optimizes pre-judgment results in combination with image recognition technology, including image binarization processing, morphological analysis, connected body feature recognition and attribute extraction, determines the direction of the parking space wireframe and the position of the vehicle through linear regression and geometric calculation, and thus judges the relative position of the parking space and the vehicle.

[0006] In a first aspect, the present invention provides a berth detection system based on a large model, comprising: An image recognition module, used to identify areas in the captured image where a parking space wireframe may exist and areas where a vehicle may exist by a cross-modal contrastive learning method, wherein the cross-modal contrastive learning method includes mapping the captured image and the description text into a common vector space by a CLIP multimodal model; An image processing module, used for screening the parking area and the vehicle area according to the similarity between the multi-modal data and performing binarization processing and morphological analysis on the screened parking area images; A connected body recognition module is used to recognize connected bodies in the berth area image and extract the properties of the connected bodies, and to secondary screen the berth area by restricting the properties of the connected bodies in the berth area image; A centroid clustering module is used to calculate the centroid of the area image and cluster the centroid of the berth area image; The position determination module is used to determine the relative position of the berth and the vehicle according to their center of mass.

[0007] Furthermore, the image recognition module outputs the captured image and the description text about the parking space and the vehicle as an image feature vector and a text feature vector respectively, and obtains the area where the parking space wireframe may exist and the area where the vehicle may exist in the captured image by matching and retrieving the image feature vector and the text feature vector. It improves the recognition accuracy by realizing cross-modal contrast learning between the image and the text.

[0008] Furthermore, the CLIP model processes the input captured image through a series of convolutional or Transformer layers and outputs a vector of a fixed size, namely the image feature vector, which is the representation of the captured image in the vector space.

[0009] Furthermore, the CLIP model passes the input description text through a text encoder, such as a Transformer, and outputs a vector of a fixed size, namely a text feature vector, which is a representation of the description text in the vector space.

[0010] Furthermore, the image processing module obtains the similarity between the captured image and the description text by normalizing the image feature vector and the text feature vector.

[0011] Furthermore, the normalization process is to divide the eigenvector by its own norm so that the norm of the eigenvector is 1.

[0012] Furthermore, the similarity between the captured image and the description text is the dot product between the feature vectors of the captured image and the description text, that is, the cosine value of the angle between the two feature vectors. The similarity ranges from -1 to 1. When the similarity is 1, the two feature vectors are exactly the same. When the similarity is -1, the two feature vectors are completely opposite. When the similarity is 0, the two are orthogonal.

[0013] Furthermore, the binarization process converts the grayscale image into a binary image, and the pixel values ​​in the binary image are represented by 0 or 1, 0 represents black, and 1 represents white.

[0014] Furthermore, the image processing module performs binarization processing on the berth image by calling the imbinarize function in MATLAB.

[0015] Furthermore, the image processing module performs binarization processing on the berth image in a manner including a global threshold method and an adaptive threshold method. The global threshold method is to set the part of all pixel values ​​in the grayscale image that are higher than the threshold to 1 and the part that are lower than the threshold to 0 by selecting a threshold. The adaptive threshold method is to dynamically adjust the threshold according to the local area of ​​the image.

[0016] Furthermore, the image processing module performs morphological analysis on the berth image through MATLAB, and the morphological analysis operates on the berth image through structural elements, and the operation methods include expansion, corrosion, opening operation, closing operation, etc., so as to achieve the purposes of target area extraction, shape analysis and image restoration.

[0017] Furthermore, the operations of morphological analysis specifically include: dilation: increasing the area of ​​the object by expanding the boundary of the object in the image outward, filling the holes in the object and connecting adjacent objects; erosion: reducing the area of ​​the object by shrinking the boundary of the object in the image inward, thereby removing small noise on the boundary of the object; opening operation: first performing erosion and then dilation, which is used to remove small noise points while retaining the overall shape of the object; closing operation: first performing dilation and then erosion, which is used to fill small holes in the object while retaining the overall shape of the object; hit-miss transformation: used to detect specific shapes or patterns.

[0018] Furthermore, the properties of the connected body extracted by the connected body identification module include area and eccentricity.

[0019] Furthermore, the connected body recognition module performs connected body recognition on the parking space image through MATLAB. Connected body recognition is to recognize connected areas or connected components in the binary image, specifically, areas composed of adjacent pixels.

[0020] Furthermore, the methods for connected body identification include: connected component labeling: assigning a unique label to each connected body in the image by calling the bwlabel and bwconncomp functions in MATLAB; region filling method: calling the imfill function in MATLAB to gradually expand the filled pixels outward from the seed point until the filled area and the background are no longer connected; connected body analysis: calling the regionprops function in MATLAB to analyze the shape, area, perimeter and other features of the identified connected bodies.

[0021] Furthermore, the area of ​​a connected body is measured by the number of speed limits contained in the connected body.

[0022] Pre-judgment is made through the CLIP model, and the results are optimized in combination with image recognition technology. Through technologies such as adaptive threshold method and morphological processing, the system can adapt to different lighting conditions and image quality, thereby improving the accuracy and robustness of recognition.

[0023] Furthermore, the centroid clustering module clusters the centroids of the parking area image through the K-means algorithm to determine the position of the parking wireframe, and determines the position of the vehicle according to the centroid of the vehicle area image. Furthermore, the steps of the K-means algorithm include: predetermining the number of classifications of the data set; randomly selecting a certain number of centroids for initialization and iterating; assigning each data point to the cluster where the nearest centroid is located according to the Euclidean distance; for each cluster, calculating the mean of all data points in the cluster and updating the position of the centroid; looping and iterating until the centroid no longer changes or reaches a predetermined number of iterations.

[0024] Its computational complexity is low, which improves the responsiveness of the system; through cluster analysis, the straight line where the long side of the berth wireframe is located can be determined more accurately, thereby improving the accuracy of berth wireframe recognition; in image registration, the uniform distribution of cluster centers in the overlapping area of ​​the image selects the optimal internal point set, which effectively enhances the image registration accuracy.

[0025] Furthermore, a straight line passing through the centroid of the vehicle area image and perpendicular to the berth direction is determined according to the orientation of the berth wireframe and the position of the vehicle.

[0026] Furthermore, the relative positions of the berth and the vehicle are determined by judging whether a straight line in the direction of the berth and a straight line passing through the centroid of the vehicle area image and perpendicular to the berth have an intersection in the captured image, and whether the intersection is located inside the berth wireframe.

[0027] Its computational complexity is low, which improves the system's responsiveness; by combining linear regression and geometric calculations, the system can better handle various environmental changes, such as different lighting conditions and complex parking scenarios, thereby improving the robustness of the system.

[0028] In a second aspect, the present invention further provides a berth detection method based on a large model, the method is based on the system described in the first aspect, and comprises: The method comprises: The cross-modal contrastive learning method is used to identify the areas where parking spaces and vehicles may exist in the captured images. Screening the parking area and the vehicle area according to the similarity between the multimodal data and performing binarization processing and morphological analysis on the screened parking area images; Conducting connected body recognition on the berth area image and extracting the attributes of the connected body; Calculating the centroid of the area image and clustering the centroid of the berth area image; After clustering, the parking space address direction and the vehicle location in the image are planned; The relative positions of the berth and the vehicle are determined based on their center of mass.

[0029] In a third aspect, the present invention further provides a computer device, comprising a processor and a memory, wherein the processor is connected to the memory, the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the computer device performs at least one step of the aforementioned large model-based berth detection method.

[0030] In a fourth aspect, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, at least one step of the aforementioned berth detection method based on a large model is implemented.

[0031] The present invention uses the CLIP model to perform comparative learning of captured images and description texts, maps the captured images and description texts into a common vector space, realizes cross-modal comparative learning, recognizes the parking space wireframe and vehicle in the captured image through the CLIP model and optimizes the pre-judgment result in combination with image recognition technology; clusters the centroid of the parking space area image to determine the straight line where the two long sides of the parking space wireframe are located, and uses the centroid of the vehicle area image as the location of the vehicle; determines the direction of the parking space wireframe and the location of the vehicle through linear regression and geometric calculation, thereby judging the relative position of the parking space and the vehicle. The system and method have a wide range of application scenarios, can realize automated high-precision parking space detection, and have low detection logic complexity.

[0032] Beneficial Effects By implementing the large model-based berth detection system and method provided by the present invention, the following technical effects are achieved: (1) The present invention uses the CLIP model to perform comparative learning between captured images and description texts, maps the captured images and description texts into a common vector space, and realizes cross-modal comparative learning. The CLIP model is used to identify parking space wireframes and vehicles in captured images and optimizes the pre-judgment results in combination with image recognition technology. By intelligently identifying parking spaces and vehicles, parking resources are more reasonably allocated to reduce the waste of parking spaces. By realizing cross-modal comparative learning between images and texts, the accuracy of recognition is improved. Pre-judgment is performed through the CLIP model and then the results are optimized in combination with image recognition technology. By using technologies such as adaptive thresholding and morphological processing, the system can adapt to different lighting conditions and image quality, thereby improving the accuracy and robustness of recognition.

[0033] (2) The centroids of the berth area images are clustered to determine the straight lines where the two long sides of the berth wireframe are located, and the centroid of the vehicle area image is used as the location of the vehicle; the computational complexity is low, and the responsiveness of the system is improved; through clustering analysis, the straight lines where the long sides of the berth wireframe are located can be determined more accurately, thereby improving the accuracy of berth wireframe recognition; in image registration, the uniform distribution of cluster centers in the overlapping area of ​​the image selects the optimal internal point set, which effectively enhances the image registration accuracy.

[0034] (3) The orientation of the parking space wireframe and the position of the vehicle are determined through linear regression and geometric calculation, thereby judging the relative position of the parking space and the vehicle. The computational complexity is low, which improves the responsiveness of the system. By combining linear regression and geometric calculation, the system can better handle various environmental changes, such as different lighting conditions and complex parking scenarios, thereby improving the robustness of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to make the above-mentioned berth detection system and method based on a large model of the present invention more obvious and easy to understand, the drawings required for use in the specific implementation of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0036] Figure 1 A schematic diagram showing the flow of a berth detection method based on a large model; Figure 2 A schematic diagram showing the technical route of the present invention; Figure 3 A schematic diagram showing the approximate determination of the berth wireframe; Figure 4A schematic diagram showing a window with a berth wireframe and other distractions; Figure 5 A schematic diagram showing a window with only the berth wireframe; Figure 6 A schematic diagram of a window showing a wireframe without a berth; Figure 7 Window schematics showing three typical examples; Figure 8 Schematic diagram of binarized images showing three typical examples; Fig. 9 Schematic diagram showing the results of morphological analysis; Fig.10 A schematic diagram showing the results of connected body identification; Fig.11 A schematic diagram showing the secondary screening berth area; Fig.12 A schematic diagram showing the centroid position of the berth area image; Fig.13 A schematic diagram showing the result of berth wireframe recognition; Fig.14 A schematic diagram showing the centroid position of the vehicle area image; Fig.15 Represents a schematic diagram summarizing features. DETAILED DESCRIPTION

[0037] Embodiment 1: A berth detection system and method based on a large model are provided. The berth detection method based on a large model is as follows Figure 1 As shown, the technical route of the present invention is as follows Figure 2 As shown, the berth detection system based on the large model includes: an image recognition module, an image processing module, a connected body recognition module, a centroid clustering module and a position judgment module, as described in detail below.

[0038] The image recognition module is used to identify areas where parking wireframes may exist and areas where vehicles may exist in the captured image through a cross-modal contrastive learning method. The cross-modal contrastive learning method includes mapping the captured image and the description text into a common vector space through a CLIP multimodal model.

[0039] The image recognition module outputs the captured image and the description text about the parking space and the vehicle as an image feature vector and a text feature vector respectively, and obtains the area where the parking space wireframe may exist and the area where the vehicle may exist in the captured image by matching and retrieving the image feature vector and the text feature vector.

[0040] The CLIP model processes the input captured image through a series of convolutional or Transformer layers and outputs a vector of a fixed size, namely the image feature vector, which is the representation of the captured image in the vector space.

[0041] The CLIP model passes the input description text through a text encoder, such as a Transformer, and outputs a vector of a fixed size, namely a text feature vector, which is the representation of the description text in the vector space.

[0042] The image processing module is used to filter the berth area and the vehicle area according to the similarity between the multimodal data and perform binarization processing and morphological analysis on the filtered berth area images.

[0043] The image processing module obtains the similarity between the captured image and the description text by normalizing the image feature vector and the text feature vector.

[0044] Normalization is to divide the eigenvector by its own norm so that the norm of the eigenvector is 1.

[0045] The similarity between the captured image and the description text is the dot product between the feature vectors of the captured image and the description text, that is, the cosine value of the angle between the two feature vectors. The similarity ranges from -1 to 1. When the similarity is 1, the two feature vectors are exactly the same. When the similarity is -1, the two feature vectors are completely opposite. When the similarity is 0, the two feature vectors are orthogonal.

[0046] Binarization is the process of converting a grayscale image into a binary image. The pixel values ​​in a binary image are represented by 0 or 1, 0 represents black and 1 represents white.

[0047] The image processing module performs binarization processing on the berth image by calling the imbinarize function in MATLAB.

[0048] The image processing module performs binarization processing on the parking image in the following ways: the global threshold method and the adaptive threshold method. The global threshold method sets the pixel values ​​of all the pixels in the grayscale image that are higher than the threshold to 1 and the pixel values ​​that are lower than the threshold to 0 by selecting the threshold. The adaptive threshold method dynamically adjusts the threshold according to the local area of ​​the image.

[0049] The image processing module performs morphological analysis on the berth image through MATLAB. Morphological analysis includes operating the image through structural elements, such as expansion, corrosion, opening operation, closing operation, etc., to achieve the purpose of target area extraction, shape analysis and image restoration.

[0050] The specific operations of morphological analysis include: dilation: increasing the area of ​​the object by expanding the boundary of the object in the image outward, filling the holes in the object and connecting adjacent objects; erosion: reducing the area of ​​the object by shrinking the boundary of the object in the image inward, thereby removing small noise on the boundary of the object; opening operation: first performing erosion and then dilation, which is used to remove small noise points while retaining the overall shape of the object; closing operation: first performing dilation and then erosion, which is used to fill small holes in the object while retaining the overall shape of the object; hit-miss transformation: used to detect specific shapes or patterns.

[0051] The connected body recognition module is used to recognize connected bodies in the berth area image and extract the properties of the connected bodies.

[0052] The attributes of the connected bodies extracted by the connected body recognition module include area and eccentricity, and the berth area is screened secondary by restricting the area and eccentricity of the connected bodies in the berth area image.

[0053] The connected body recognition module performs connected body recognition on the parking space image through MATLAB. Connected body recognition is to recognize connected areas or connected components in the binary image, specifically, areas composed of adjacent pixels.

[0054] The methods for identifying connected bodies include: connected component labeling: assigning a unique label to each connected body in the image by calling the bwlabel and bwconncomp functions in MATLAB; region filling method: calling the imfill function in MATLAB to gradually expand the filled pixels outward from the seed point until the filled area and the background are no longer connected; connected body analysis: calling the regionprops function in MATLAB to analyze the shape, area, perimeter and other features of the identified connected bodies.

[0055] The area of ​​a connected body is measured by the number of speed limits contained in the connected body.

[0056] The centroids of the berth area image and the vehicle area image are the center points of the area where the berth wireframe may exist and the area where the vehicle may exist.

[0057] The centroid clustering module is used to calculate the centroid of the area image and cluster the centroid of the berth area image.

[0058] The centroid clustering module clusters the centroids of the berth area image through the K-means algorithm to determine the position of the berth wireframe, and determines the position of the vehicle according to the centroid of the vehicle area image.

[0059] The steps of the K-means algorithm include: predetermining the number of categories of the data set; randomly selecting a certain number of centroids for initialization and iterating; assigning each data point to the cluster where the nearest centroid is located according to the Euclidean distance; for each cluster, calculating the mean of all data points in the cluster and updating the position of the centroid; and looping and iterating until the centroid no longer changes or a predetermined number of iterations is reached.

[0060] The position determination module is used to determine the relative position of the berth and the vehicle according to their centroids.

[0061] A straight line passing through the centroid of the vehicle area image and perpendicular to the berth direction is determined according to the orientation of the berth wireframe and the position of the vehicle.

[0062] The relative position of the berth and the vehicle is determined by judging whether a straight line in the direction of the berth and a straight line passing through the centroid of the vehicle area image and perpendicular to the berth have an intersection in the captured image, and whether the intersection is located inside the berth wireframe.

[0063] Embodiment 2: On the basis of the above-mentioned embodiments, three typical cases are taken as examples to describe the process of implementing the berth detection method based on a large model.

[0064] The CLIP library is called and the scanning window width is set to 100 to identify the area where the parking line frame may exist in the captured image.

[0065] By calling the CLIP model, the captured image and the description text of the parking space and the vehicle are output as a fixed-size feature vector and the feature vector is normalized to obtain the similarity between the captured image and the description text. By setting the threshold of the obtained similarity to 0.23, the green frames with similarity less than the threshold are filtered out, and the remaining windows are retained as red frames. Figure 3 shown.

[0066] There are windows with berth wireframes and other distractions such as Figure 4 As shown in the figure, it contains both the parking frame and other interference images; only the window of the parking frame is as shown in the figure. Figure 5 As shown in the figure, it only contains pure berth wireframe, with a single target, which is relatively easy to identify; the window without berth wireframe is shown in the figure. Figure 6 shown, without the berth wireframe.

[0067] The window diagrams of three typical examples are as follows: Figure 7 As shown in the figure, the contents of the window are captured, and each window is a small image of size 100*100.

[0068] The binary images of three typical examples are as follows Figure 8As shown in the figure, the results of three example window images after binarization are shown.

[0069] The morphological analysis results are as follows Fig. 9 As shown, Figure 8 The image on the right side of the middle obviously does not have a continuous strip frame, so morphological analysis cannot be performed. Therefore, only the left and middle images are analyzed, and a disk with a radius of 3 pixels is used to fill the image. Figure 8 The strip holes in the left image and the middle image in the figure give Fig. 9 result.

[0070] according to Fig. 9 After morphological analysis, the image is used to identify the connected body area and calculate the coordinates of the connected body area. The connected body identification results are as follows: Fig.10 As shown, the area within the red frame is the identified connected body area.

[0071] According to the connected body properties, the berth areas that do not meet the requirements are screened again. The screening conditions are that the eccentricity is greater than 0.8 and the area is greater than 100. The berth areas after the secondary screening are as follows: Fig.11 As shown, there are 8 windows in total.

[0072] according to Fig.11 The red box shown in the figure calculates the centroid of the berth area image and obtains the coordinates of the centroid of the berth area image. The centroid position of the berth area image is as follows: Fig.12 shown.

[0073] Berth wireframe recognition results are as follows Fig.13 As shown, according to the clustering results, the centroids 1, 2 and 3 are obtained by linear regression. Fig.13 The green fitting line in the middle is obtained by linear regression of the centroids 4, 5, 6, 7 and 8. Fig.13 The two fitted straight lines in medium blue roughly indicate the direction of the berth wireframe.

[0074] By calling the CLIP library and setting the scanning window width to 250, the area where the vehicle may exist in the captured image is identified. The centroid position of the vehicle area image is as follows: Fig.14 As shown, the window with the highest similarity, that is, the area in the red box, is taken as the result of this vehicle position recognition.

[0075] The schematic diagram of the element summary is as follows Fig.15 As shown, Fig.13 Take the middle line of the two green and blue lines representing the parking space wireframe to get a yellow straight line. The yellow straight line represents the average direction of the parking space. Draw a purple straight line in the orthogonal direction of the yellow straight line through the green centroid representing the vehicle position. The purple straight line indicates the direction of the vehicle position perpendicular to the parking space.

[0076] The yellow straight line and the purple straight line represent the average direction of the berth and the direction perpendicular to the berth, respectively. Solve the intersection of the two straight line equations. If the intersection exists and is within the berth range, the vehicle is determined to be on the berth. Otherwise, the vehicle is determined to be outside the berth. The equations of the two straight lines are as follows:

[0077]

[0078] Finally, the intersection coordinates are obtained as (272.6519, 488.7201), which is within the berth range, so it is determined that the vehicle is located in the berth.

[0079] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable non-transient storage media containing computer-usable program code.

[0080] The present invention can provide computer program instructions to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce means for implementing the system.

[0081] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions of the system.

[0082] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions of the system.

Claims

1. A berth detection system based on a large model, characterized in that: include: An image recognition module, used to identify areas in the captured image where a parking space wireframe may exist and areas where a vehicle may exist by using a cross-modal contrastive learning method, wherein the cross-modal contrastive learning method includes mapping the captured image and the description text into a common vector space by using a CLIP multimodal model; An image processing module, used for screening the parking area and the vehicle area according to the similarity between the multi-modal data and performing binarization processing and morphological analysis on the screened parking area images; A connected body recognition module is used to recognize connected bodies in the berth area image and extract the properties of the connected bodies, and to secondary screen the berth area by restricting the properties of the connected bodies in the berth area image; A centroid clustering module is used to calculate the centroid of the area image and cluster the centroid of the berth area image; The position determination module is used to determine the relative position of the berth and the vehicle according to their center of mass.

2. The system according to claim 1, characterized in that: The image recognition module outputs the captured image and the description text about the parking space and the vehicle as an image feature vector and a text feature vector respectively, and obtains the area where the parking space wireframe may exist and the area where the vehicle may exist in the captured image by matching and retrieving the image feature vector and the text feature vector.

3. The system according to claim 2, characterized in that: The image processing module obtains the similarity between the captured image and the description text by normalizing the image feature vector and the text feature vector.

4. The system according to claim 3, characterized in that: The similarity between the captured image and the description text is the dot product between the image feature vector and the text feature vector.

5. The system according to claim 4, characterized in that: The properties of the connected body extracted by the connected body identification module include area and eccentricity.

6. The system according to claim 1, characterized in that: The centroid clustering module clusters the centroids of the berth area image through the K-means algorithm to determine the position of the berth wireframe, and determines the position of the vehicle according to the centroid of the vehicle area image.

7. The system according to claim 6, characterized in that: The relative position of the berth and the vehicle is determined by judging whether a straight line in the direction of the berth and a straight line passing through the centroid of the vehicle area image and perpendicular to the berth have an intersection in the captured image, and whether the intersection is located inside the berth wireframe.

8. A berth detection method based on a large model, characterized in that: The method is implemented based on the system according to any one of claims 1 to 7: The method comprises: The cross-modal contrastive learning method is used to identify the areas where parking spaces and vehicles may exist in the captured images. Screening the parking area and the vehicle area according to the similarity between the multimodal data and performing binarization processing and morphological analysis on the screened parking area images; Conducting connected body recognition on the berth area image and extracting the attributes of the connected body; Calculating the centroid of the area image and clustering the centroid of the berth area image; After clustering, the parking space address direction and the vehicle location in the image are planned; The relative positions of the berth and the vehicle are determined based on their center of mass.

9. A computer device comprising a processor and a memory, wherein the processor is connected to the memory, and the memory is used to store a computer program, wherein: The processor is configured to execute the computer program stored in the memory, so that the computer device performs at least one step of the berth detection method based on a large model as claimed in claim 8.

10. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, characterized in that: When the computer program is executed, at least one step of the berth detection method based on a large model as claimed in claim 8 is implemented.

Images