Parking space marking method and device based on image, equipment and medium

Abstract

The invention discloses an image-based parking space marking method and device, equipment and a medium. The parking space marking method comprises the following steps: cutting regions of interest of a parking space in a look-around image, and generating an inquiry text of each region of interest; and inputting each region of interest and the corresponding inquiry text into a pre-trained parking space attribute determination model, and obtaining a parking space attribute file output by the pre-trained parking space attribute determination model.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image recognition, and in particular to a parking space labeling method and device based on images, equipment and medium. BACKGROUND

[0002] The labeling method of parking space attributes mainly includes manual labeling and automatic labeling. Manual labeling requires labeling personnel to fully understand and follow the labeling specification on the surround view, and then assign corresponding attributes to each parking space one by one. Although this method has high accuracy, it is expensive and inefficient. Automatic labeling relies on lane lines and obstacle information, and codes the labeling process by writing rules or algorithms, which can significantly improve efficiency. However, due to the difficulty of rules to cover all scenarios, misjudgments often occur in corner parking spaces and other boundary situations. Therefore, it is urgent to design a more robust automatic labeling pipeline that retains high efficiency and reduces errors in corner parking spaces and other extreme scenarios. The method proposed in this paper aims to solve the contradiction between high cost and high efficiency in parking space labeling. SUMMARY

[0003] The present application mainly provides a parking space labeling method and device based on images to solve the problem of balancing the cost and efficiency of parking space labeling.

[0004] To solve the above technical problems, one technical solution adopted by the present application is to provide a parking space labeling method based on images, which includes: cropping the region of interest of the parking space in the surround view image, and generating the query text of each region of interest; inputting each region of interest and the corresponding query text into a pre-trained parking space attribute determination model to obtain the parking space attribute file output by the pre-trained parking space attribute determination model.

[0005] In some embodiments, the parking space labeling method further includes: inserting a small adaptation module into the key layer of an initial parking space attribute determination model; and only pre-training the parameters of the small adaptation module based on artificial labeling data to obtain a pre-trained parking space attribute determination model.

[0006] In some embodiments, the pre-training of the parameters of the small adaptation module based on artificial labeling data includes: taking the artificial labeling data as input to obtain the parking space attribute file output by the initial parking space attribute determination model; and pre-training the parameters of the small adaptation module based on the loss function, the parking space attribute file and the artificial labeling data, wherein the loss function includes a backbone loss, an attribute classification loss and a confidence regression loss.

[0007] In some embodiments, before the obtaining the parking space attribute file output by the pre-trained parking space attribute determination model, the method further comprises: obtaining multiple frames of continuous parking space attributes within a preset time window, and performing smoothing processing on the parking space attributes of each parking space within the time window.

[0008] In some embodiments, the parking space attributes of the parking space attribute file comprise parking space types, parking space opening directions, and parking space occupancy states, and the parking space attribute file further comprises confidences corresponding to each parking space attribute.

[0009] In some embodiments, after the obtaining the parking space attribute file output by the pre-trained parking space attribute determination model, the method further comprises: filtering the parking space attribute file based on the confidences, to retain the parking space attribute file whose confidence corresponding to each parking space attribute reaches a preset confidence threshold.

[0010] In some embodiments, the parking space labeling method further comprises: marking the parking space attribute file whose confidence is less than the preset confidence threshold as a review sample; and performing real-time incremental fine-tuning on the pre-trained parking space attribute determination model according to the review sample after manual labeling.

[0011] In some embodiments, the cropping of the region of interest of each parking space in the surround view image comprises: obtaining a real-time surround view image stream, and performing preprocessing on the surround view image in the surround view image stream; detecting corner coordinates of each parking space in the preprocessed surround view image through a corner detection module; and cropping the region of interest of each parking space in the surround view image based on the corner coordinates of each parking space.

[0012] To solve the above technical problems, another technical solution adopted by the present application is to provide a parking space labeling device based on images, comprising: a cropping module configured to crop the region of interest of each parking space in a surround view image, and generate query texts of each region of interest; and an output module configured to input each region of interest and the corresponding query text into a pre-trained parking space attribute determination model, and obtain a parking space attribute file output by the pre-trained parking space attribute determination model.

[0013] The present application further provides a computer device, comprising: a memory and at least one processor, wherein the memory stores instructions; and the at least one processor invokes the instructions in the memory, so that the computer device performs the parking space labeling method as described above.

[0014] The present application further provides a computer readable storage medium, wherein the computer readable storage medium stores instructions, and the instructions are executed by a processor to implement the parking space labeling method as described above.

[0015] The beneficial effects of the present application are: different from the prior art, the present application discloses a parking space marking method, device, equipment and medium based on image, the interested region of the parking space is cropped in the surround view image, and the inquiry text of each interested region is generated; according to the coordinates provided by the upstream module, the small image containing only one candidate parking space and its adjacent peripheral region is accurately cut out from the complete surround view bird's eye view, the macroscopic problem of processing the global complex scene is converted into the microscopic problem of analyzing one by one local simple images. The interference of irrelevant background information is greatly shielded, and the subsequent model only focuses on the most critical features of the parking space. The parking space attribute judgment model pre-trained by artificial mark data is input into each interested region and the corresponding inquiry text, and the parking space attribute file output by the pre-trained parking space attribute judgment model is obtained; each interested region and the corresponding inquiry text are input into the pre-trained parking space attribute judgment model, the model understands the problem based on image content, and generates a text answer, and the system parses the answer into a structured parking space attribute file, based on the geometric shape of the parking space, the relationship of the surrounding vehicles, the relative position with the ego vehicle and other information, the parking space attribute is comprehensively inferred, and the parking space attribute judgment is efficiently and accurately completed. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor based on these drawings. Figure 1 is a flowchart of an embodiment of the parking space marking method based on image provided by the present application; Figure 2 is a flowchart of an embodiment of the method step 10 shown in Figure 1 ; Figure 3 is a flowchart of another embodiment of the method shown in Figure 1 ; Figure 4 is a flowchart of an embodiment of the method step 40 shown in Figure 3 ; Figure 5 is a flowchart of another embodiment of the method shown in Figure 1 ; Figure 6 is a structural diagram of an embodiment of the parking space marking device based on image provided by the present application; Figure 7 is a structural diagram of an embodiment of the computer equipment in the embodiments of the present application. DETAILED DESCRIPTION

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0018] The terms "first," "second," and "third" used in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first," "second," or "third" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0019] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0020] See Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the image-based parking space labeling method provided in this application. The image-based parking space labeling method includes the following steps: 10: Crop the regions of interest for parking spaces in the panoramic image and generate query text for each region of interest.

[0021] After the panoramic images are calibrated and stitched together to obtain a complete panoramic view, the four corner points of the parking spaces are located using an existing corner detection module. Based on this, a local image region containing a single parking space, i.e., the region of interest, is cropped from the panoramic image. Structured text prompts are automatically generated for each parking space's region of interest, containing queries about the parking space's attributes.

[0022] For example, please determine the type of parking space (perpendicular / parallel / diagonal), the direction of the opening (left / right / up / down / both directions), and the vacancy status (vacant / occupied).

[0023] The surround view image is an image of 360° panoramic surround view formed by stitching four fisheye cameras or multi-directional pinhole cameras, etc. alone or in combination, after distortion correction and external parameter alignment, which can cover the environment around the vehicle.

[0024] The region of interest is a local image region containing a single parking space cropped from the panoramic surround view image, which is the core content that the focus model needs to analyze, reducing irrelevant information interference.

[0025] The query text is a structured text prompt designed for parking space attributes, which is used to guide the visual-language model to output structured results and realize the alignment of image and semantics.

[0026] By automatically cropping the region of interest and generating the query text, manual region framing and attribute description for each parking space are avoided, laying the foundation for subsequent large-scale reasoning, and batch processing data at one time. At the same time, the structured query text clearly defines the attribute category, guides the model to output standardized results, and reduces ambiguity; combined with the focus of the local image region, the influence of background interference on model judgment is reduced. There is no need for manual rule writing or manual annotation of attribute description for each parking space.

[0027] Further, referring to Figure 2 , step 10 further comprises the following steps: 11: Obtain a real-time surround view image stream, and pre-process the surround view images in the surround view image stream.

[0028] Real-time image data is collected by a panoramic camera mounted on the vehicle to form a surround view image stream; each frame of image is de-distorted, aligned with external parameters, stitched into an aerial view, and down-sampled to reduce the amount of calculation.

[0029] The surround view image stream is a continuous image sequence collected by the camera in real time, covering the 360° environment around the vehicle, providing original visual data for parking space detection.

[0030] Optionally, the surround view image stream can be collected by fisheye cameras arranged around the vehicle.

[0031] Specifically, the pre-processing of the surround view images in the surround view image stream includes: Distortion correction: correcting the image distortion caused by the fisheye camera lens to restore the true scene scale; external parameter alignment: unifying the spatial coordinate systems of the four cameras to ensure consistent viewing angles when stitching images; stitching into an aerial view: combining four images into a bird's eye view centered on the vehicle; down-sampling: compressing the stitched image by 4-8 times resolution to reduce the consumption of computing resources in subsequent processing.

[0032] 12: Detect the corner coordinates of each parking space in the pre-processed surround view image through a corner detection module.

[0033] The corner point detection module is used to identify and locate the four corner point coordinates of each parking space in the pre-processed surround view image, and determine the specific contour position of the parking space in the image.

[0034] The corner point detection module is a functional module based on computer vision algorithms such as Harris corner detection and edge feature extraction. It identifies the corner features of the parking space boundary in the image, i.e. the corner points, and outputs the four corner point coordinates of the parking space contour for positioning the parking space area.

[0035] Pre-processing eliminates image distortion, unifies the perspective, and compresses the data volume to ensure the accuracy and real-time performance of subsequent parking space detection, avoiding interference from noise or redundant information in the original image. The corner point detection module identifies the corner coordinates of the parking space to provide an accurate spatial range for subsequent cropping of the parking space region of interest, ensuring that each parking space area is independently and completely extracted.

[0036] Steps 11 to 12 convert the original image into structured parking space contour data, which is a prerequisite for subsequent steps such as generating query text models for reasoning, and promotes the transition from manual dependence to full automation of the labeling process. Reusing existing corner point detection modules saves costs, while optimizing input data through pre-processing improves the reasoning efficiency and robustness of downstream modules.

[0037] 13: Based on the corner coordinates of each parking space, crop the region of interest of each parking space from the surround view image.

[0038] Based on the detected corner coordinates of the parking space, crop the region of interest containing a single parking space from the surround view image, focusing on the core area for subsequent model analysis.

[0039] The region of interest cropping focuses on the single parking space area, eliminating irrelevant background pixels, so that the downstream modules only need to process local key information, reducing the amount of calculation and improving the reasoning speed. This process converts the original image into structured parking space region of interest data, providing standardized input for subsequent steps, promoting full-process automation from image acquisition to attribute labeling, and reducing the cost of manual intervention.

[0040] 20: Input each region of interest and the corresponding query text into the pre-trained parking space attribute judgment model to obtain the parking space attribute file output by the pre-trained parking space attribute judgment model.

[0041] The cropped parking space region of interest and the corresponding structured query text are input into the pre-trained and fine-tuned parking space attribute judgment model. The model fuses image visual features and text semantic information to output a structured parking space attribute file containing parking space type, opening direction, occupancy status, and confidence.

[0042] The pre-trained parking attribute determination model is based on an open-source visual language large model, such as BLIP-2, MiniGPT-4, etc., and is obtained through a small amount of artificial fine label data for LoRA light fine-tuning, and has the ability to simultaneously understand image texture features and text semantics, and can directly output structured results and confidence of parking attribute.

[0043] The pre-trained parking attribute determination model is based on an open-source visual language large model, such as BLIP-2, MiniGPT-4, etc., and is obtained through a small amount of artificial fine label data for LoRA light fine-tuning, and has the ability to simultaneously understand image texture features and text semantics, and can directly output structured results and confidence of parking attribute.

[0044] In the LoRA light fine-tuning process, the model backbone parameters are frozen, and only the low-rank adaptation layer is fine-tuned to quickly adapt to the parking attribute determination task with a small amount of artificial label data, reducing the calculation cost.

[0045] The model directly takes the region of interest image and the query text as input, and outputs the structured attribute results, without human intervention in the middle link, greatly improving the labeling efficiency. Only a small amount of initial artificial labeling data is needed to complete model fine-tuning, and subsequent batch reasoning is completely automated, avoiding the maintenance cost and artificial review cost of a large number of handwritten rules in traditional methods.

[0046] Optionally, the parking attribute of the parking attribute file includes parking type, parking opening direction, and parking empty state, and the parking attribute file further includes confidence corresponding to each parking attribute.

[0047] The parking attribute file includes parking type, such as vertical, parallel, and diagonal, and its confidence, i.e., the reliability score of the model's type determination; the parking opening direction, such as left, right, up, and down, and its confidence; and the parking empty state, such as empty and occupied, and its confidence.

[0048] This structured design can directly support subsequent filtering based on confidence and decision-making needs of downstream business systems.

[0049] Optionally, before obtaining the parking attribute file output by the pre-trained parking attribute determination model, there is further comprising obtaining multiple frames of continuous parking attributes within a preset time window, and performing smoothing processing on the parking attributes of each parking within the time window.

[0050] Within a preset time window, multiple frames of attribute detection results of the same parking are continuously obtained, and the multiple frames of results are fused through a smoothing algorithm, and after eliminating single-frame noise interference, a final stable parking attribute file is output.

[0051] For example, in the last 5 frames of images, corresponding to about 1 second, multiple frames of empty state of the same parking space are continuously acquired, the results of multiple frames are fused through time sliding average method or median filtering or voting mechanism, and after eliminating single frame noise interference, the final stable parking space attribute file is output.

[0052] The preset time window is a continuous time interval (such as 500 ms to 2 s) set by a person, which is used to collect multiple frame attribute data of the same parking space, and balance real-time and smoothing effect. If the window is too short, it may not be able to filter transient noise, and if it is too long, it may delay state update. In the time window, the attribute result sequence of the same parking space detected multiple times, such as "occupied, occupied, empty, occupied, occupied", contains random errors or transient interference of single frame detection, such as misjudgment caused by shadow or temporary occlusion.

[0053] By fusing multiple frame data, filtering single frame misjudgment caused by sudden changes in light, temporary occlusion of vehicles, camera shaking, etc., the attribute result is closer to the real state. Avoid frequent jumping between empty and occupied of parking space attribute, output smoother result, reduce invalid state update of downstream system.

[0054] Referring to Figure 3 In another embodiment, the image-based parking space labeling method further comprises the following steps: 30: Insert a small adaptation module in the key layer of the initial parking space attribute determination model.

[0055] In the key layer of the model, such as the high layer of the feature extraction network, the cross-modal fusion layer or the bottleneck layer before the classification head, a lightweight adaptation module such as Adapter, convolution attention module is inserted, and by training only the parameters of the module, the model is adapted to specific scenes such as complex light or irregular parking spaces, while avoiding the high computational cost of full parameter fine-tuning.

[0056] The initial parking space attribute determination model is the basic model before the adaptation module is inserted, which is usually constructed based on a visual language large model and includes three core layers of feature extraction, cross-modal fusion and attribute output, and can preliminarily realize parking space attribute determination but has limited generalization ability.

[0057] The key layer is the intermediate layer in the model that has the greatest impact on the final output. Optionally, the key layer includes but is not limited to the high layer of visual features, the cross-modal fusion layer or the bottleneck layer before the classification head.

[0058] The small adaptation module is a lightweight network unit with a parameter amount of only 1% to 5% of the original model, and common types include: Adapter module, convolution attention module, dynamic routing module, etc. Only training the small adaptation module can reduce the consumption of computing resources compared to full parameter fine-tuning, and support fast adaptation on edge devices.

[0059] By capturing subtle features of specific scenarios through small adaptation modules, the accuracy of attribute determination in non-standard scenarios is improved. When new parking scenarios emerge, only a small number of samples need to be collected and the small adaptation modules fine-tuned, without retraining the entire model, thus adapting to the needs of rapid iteration.

[0060] 40: Only the parameters of the small adaptation module are pre-trained based on manually labeled data to obtain the pre-trained parking space attribute determination model.

[0061] With the core parameters of the initial parking space attribute determination model frozen, the parameters of the inserted small adaptation modules are specifically trained using only manually labeled parking space attribute data. This allows the modules to learn the parking space feature distribution of a specific scenario, ultimately resulting in a pre-trained model adapted to the target scenario.

[0062] By training the adaptation module parameters alone, the initial model is adapted to the final model for a specific scenario. The core parameters of the model retain the general feature extraction capabilities from the pre-training stage, while the adaptation module learns the specific features of the target scenario, achieving the advantage of combining general capabilities with scenario adaptation.

[0063] See Figure 4 Furthermore, step 40 also includes the following steps: 41: Using manually labeled data as input, obtain the parking space attribute file output by the initial parking space attribute determination model.

[0064] Using manually labeled data as input, images are fed into the visual encoder of the initial parking space attribute determination model, while text queries are fed into the text encoder. After processing by a cross-modal fusion layer, the classification head outputs a preliminary parking space attribute file. During this process, the model's backbone parameters are frozen, and only a small adaptation module participates in forward propagation, outputting unoptimized initial prediction results.

[0065] 42: Based on parking space attribute files and manually labeled data, calculate loss functions to pre-train the parameters of small adaptation modules. The loss functions include backbone loss, attribute classification loss, and confidence regression loss.

[0066] Based on the parking space attribute file and manually labeled data output in step 41, the prediction error is calculated using the loss function, the gradient is backpropagated, and the parameters of the small adaptation module are updated to achieve model fine-tuning.

[0067] The backbone loss is a loss that constrains the general capabilities of the model during pre-training, and it typically reuses the loss function from the original pre-training task. When freezing the backbone parameters, a small amount of gradient backpropagation is used to prevent the backbone feature extraction capability from degrading, thus maintaining the model's understanding of basic visual language alignment.

[0068] The attribute classification loss is calculated by cross-entropy loss or focal loss for discrete attributes such as parking space type, opening direction, and occupancy state, to calculate the error between the predicted class and the true label. It can optimize the discrimination ability of small adaptive modules for specific attributes, such as focusing on difficult-to-classify samples such as diagonal parking spaces and bidirectional openings through focal loss, to improve the recognition accuracy of sparse attributes.

[0069] The confidence regression loss is calculated by mean squared error or Huber loss for the attribute confidence output by the model, to align the predicted confidence with the true classification accuracy. It avoids outputting high-confidence incorrect results by the model, improves the reliability of the attribute file, and provides a reliable basis for decision-making for downstream systems.

[0070] The three types of loss functions work together to constrain the small adaptive module to quickly learn the scene-specificity of the manually labeled data, improving the parameter update efficiency.

[0071] Optionally, after step 20, step 50 is further included: filtering the parking space attribute file based on the confidence to retain the parking space attribute file whose confidence corresponding to each parking space attribute reaches a pre-set confidence threshold.

[0072] A pre-set confidence threshold is set, and the parking space attribute file output by step 41 is traversed to filter out the parking space attribute file whose predicted confidence of each parking space attribute is not less than the threshold, and the parking space attribute file whose predicted confidence of each parking space attribute is less than the threshold is marked as low confidence and triggers subsequent processing. This process filters unreliable predictions through quantitative indicators, improving the quality of input data for downstream systems.

[0073] The confidence threshold set according to business needs is used to distinguish between reliable predictions and unreliable predictions.

[0074] For example, the threshold is set to 0.8. Only the results that the model is highly confident in are retained, sacrificing part of the recall rate to ensure the accuracy rate. For example, the threshold is set to 0.6. More low-confidence results are allowed to pass, which is suitable for scenarios that require comprehensive coverage of parking space status.

[0075] Filtering low-confidence predictions reduces the decision-making errors of downstream systems caused by model misjudgment, and improves the accuracy of effective attribute files.

[0076] Referring to Figure 5 , step 50 further includes the following steps: 51: Mark the parking space attribute file with a confidence less than the pre-set confidence threshold as a review sample.

[0077] With the results filtered in step 43 as input, all parking space attribute files are traversed: if the confidence of any key attribute in the file is lower than the preset confidence threshold, the file is marked as a review sample and stored in the manual annotation queue. When marking, the specific field of the low-confidence attribute and the corresponding parking space region of interest image are recorded for targeted review by the annotator.

[0078] 52: Real-time incremental fine-tuning of the pre-trained parking attribute determination model based on the review samples after manual annotation.

[0079] After the review samples in the manual annotation queue are corrected, the newly annotated data is used as an incremental training set to fine-tune the pre-trained parking attribute determination model again. During fine-tuning, the model backbone parameters are still frozen, and only the small adaptive module parameters are optimized to enable the model to quickly learn the error patterns in low-confidence samples and achieve dynamic iterative optimization.

[0080] The targeted fine-tuning of low-confidence samples improves the model's recognition accuracy for similar errors; only low-confidence samples need to be annotated, reducing the workload of manual annotation and avoiding resource waste caused by full retraining.

[0081] The above describes the image-based parking annotation method in the embodiments of the present application, and the following describes the image-based parking annotation device in the embodiments of the present application. Please refer to Figure 6 An embodiment of the image-based parking annotation device in the embodiments of the present application includes: The cropping module 210 is configured to crop the region of interest of the parking space in the surround view image and generate query text for each region of interest.

[0082] The output module 220 is configured to input each region of interest and the corresponding query text into the pre-trained parking attribute determination model to obtain the parking attribute file output by the pre-trained parking attribute determination model.

[0083] The above Figure 6 The feature extraction device in the embodiments of the present application is described in detail from the perspective of modular functional entities, and the computer device in the embodiments of the present application is described in detail from the perspective of hardware processing.

[0084] Figure 7Fig. 1 is a structural schematic diagram of a computer device according to an embodiment of the present application. The computer device 500 can have great differences due to different configurations and performances, and can include one or more central processing units (CPUs) 510 (for example, one or more processors) and a memory 520, and one or more storage media 530 (for example, one or more mass storage devices) storing application programs 533 or data 532. The memory 520 and the storage media 530 can be temporary storage or persistent storage. The programs stored in the storage media 530 can include one or more modules (not shown in the figure), each of which can include a series of instruction operations on the computer device 500. Further, the processor 510 can be configured to communicate with the storage media 530 and execute a series of instruction operations in the storage media 530 on the computer device 500.

[0085] The computer device 500 can further include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input and output interfaces 560, and / or one or more operating systems 531, such as Windows Server, MacOS X, Unix, Linux, FreeBSD, etc. Those skilled in the art can understand that the computer device 500 can include more or fewer components than those shown in the figure, or some components can be combined, or different components can be arranged. Figure 7 The computer device structure shown does not constitute a limitation on the computer device, and can include more or fewer components than those shown in the figure, or some components can be combined, or different components can be arranged.

[0086] The present application also provides a computer device including a memory and a processor, the memory storing computer readable instructions, and the computer readable instructions being executed by the processor to cause the processor to perform the steps of the image-based parking space labeling method in each of the embodiments.

[0087] The present application also provides a computer readable storage medium, which can be a non-volatile computer readable storage medium or a volatile computer readable storage medium, and the computer readable storage medium stores instructions, and the instructions, when executed on a computer, cause the computer to perform the steps of the image-based parking space labeling method.

[0088] Different from the prior art, the application adopts a small amount of artificial fine mark data to fine-tune the parking space attribute determination model, so as to output a structured parking space attribute file and a confidence, instead of the traditional method relying on rules, geometric reasoning or complete artificial marking. The parking space attribute determination model simultaneously fuses image texture and language semantics, and has higher adaptation capability to various special scenes; and only a small amount of artificial marking data is required for pre-training, and subsequent automatic learning and fine-tuning, thereby reducing the marking and learning cost. The application also provides a method for automatically cropping a parking space region of interest and generating a structured query text, which replaces manual frame selection of a parking space and attribute description through an automatic process, and outputs a structured constraint format, thereby reducing the marking cost and avoiding subjective differences in marking. The application also improves the accuracy of the model in complex scenes through a confidence filtering and self-learning closed-loop mechanism, and avoids pollution of the database by low-quality samples. Through the combination of the large model light fine-tuning, the automatic process and the self-learning closed-loop technology, the scheme breaks through the bottleneck of high cost and low efficiency of traditional artificial marking, poor robustness and difficult maintenance of pure rule reasoning, realizes the paradigm upgrade of parking space marking from artificial dominance to man-machine cooperation, simultaneously supports dynamic scene adaptation and continuous performance optimization, and provides an efficient solution for large-scale landing of an automatic driving parking system.

[0089] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, for the storage medium embodiment and the computer device embodiment, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0090] The application can be used in many general or special vehicle-mounted computing system environments or configurations. For example: personal computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, network PCs, small computers, distributed computing environments including any of the above systems or devices, and the like.

[0091] In several embodiments provided in the application, it should be understood that the disclosed method and device can be implemented in other ways. For example, the device embodiment described above is only illustrative, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0092] The units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment scheme.

[0093] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0094] The above is only an embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation using the content of the present application specification and drawings, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. An image-based parking space labeling method, characterized in that, The parking space marking method includes: Cropping the regions of interest for parking spaces from the panoramic image and generating query text for each region of interest; Input each region of interest and the corresponding query text into the pre-trained parking space attribute determination model, and obtain the parking space attribute file output by the pre-trained parking space attribute determination model.

2. The parking space marking method according to claim 1, characterized in that, The parking space marking method also includes: Insert small adaptation modules into the key layers of the initial parking space attribute determination model; The parameters of the small adaptation module are pre-trained based on manually labeled data to obtain a pre-trained parking space attribute determination model.

3. The parking space marking method according to claim 2, characterized in that, The pre-training of the parameters of the small adaptation module based on manually labeled data includes: Using the manually labeled data as input, obtain the parking space attribute file output by the initial parking space attribute determination model; Based on the parking space attribute file and the manually labeled data, a loss function is calculated to pre-train the parameters of the small adaptation module. The loss function includes backbone loss, attribute classification loss, and confidence regression loss.

4. The parking space marking method according to claim 1, characterized in that, The parking space attribute file includes parking space type, parking space opening direction, and parking space vacancy status. The parking space attribute file also includes the confidence level corresponding to each of the parking space attributes.

5. The parking space marking method according to claim 4, characterized in that, After obtaining the parking space attribute file output by the pre-trained parking space attribute determination model, the method further includes: The parking space attribute files are filtered based on the confidence level to retain those parking space attribute files whose confidence levels for each parking space attribute reach a preset confidence threshold.

6. The parking space marking method according to claim 5, characterized in that, The parking space marking method also includes: The parking space attribute files with a confidence level lower than a preset confidence threshold are marked as review samples; The pre-trained parking space attribute determination model is incrementally fine-tuned in real time based on the manually labeled verification samples.

7. The parking space marking method according to claim 1, characterized in that, The process of cropping the region of interest for the parking space in the surround view image includes: Acquire a real-time panoramic image stream and preprocess the panoramic images in the panoramic image stream; The corner detection module detects the corner coordinates of each parking space in the preprocessed surround view image. Based on the corner coordinates of each parking space, the region of interest for each parking space is cropped from the panoramic image.

8. An image-based parking space marking device, characterized in that, include: The cropping module is used to crop the regions of interest for parking spaces in the surround view image and generate query text for each region of interest; The output module is used to input each of the regions of interest and the corresponding query text into the pre-trained parking space attribute determination model, and obtain the parking space attribute file output by the pre-trained parking space attribute determination model.

9. A computer device, characterized in that, The computer device includes: a memory and at least one processor, wherein the memory stores instructions; The at least one processor invokes the instructions in the memory to cause the computer device to perform the parking space marking method as described in any one of claims 1-7.

10. A computer-readable storage medium storing instructions thereon, characterized in that, When the instruction is executed by the processor, it implements the parking space marking method as described in any one of claims 1-7.

Images