AI parking stall state detection method based on multimode data

Abstract

The invention discloses an AI parking space state detection method based on multimode data, and the method comprises the steps: collecting historical multimode data and same-period image data of a target parking space, building a training sample set after marking, inputting a pre-training model, and outputting the calibration weight of each sub-mode; multi-modal data and environment parameters are collected in real time, weighted calculation is carried out after weight is dynamically adjusted, and the state of no vehicle, standard parking or non-standard parking is judged according to a score interval; and if the integrity detection is not qualified, the image is complemented based on the calibration weight in combination with the multi-view and color features, and the state is verified. The method solves the problems that the prior art is susceptible to environmental interference, depends on image data and is low in detection accuracy, improves the robustness and environmental adaptability of parking space state detection, can achieve accurate recognition without images, and reduces the labor cost.

Description

Technical Field

[0001] This invention belongs to the field of parking technology, specifically relating to an AI-based method for detecting parking space status based on multi-modal data. Background Technology

[0002] With the acceleration of urbanization and the continuous growth of motor vehicle ownership, the contradiction between the supply and demand of parking space resources is becoming increasingly prominent. Efficient and accurate parking space status detection is a core link in improving the utilization rate of parking resources and optimizing traffic management efficiency. Existing parking space status detection technologies are mainly divided into two categories: single-modal detection and multi-modal data fusion detection. Among them, in single-modal detection technology, image recognition technology collects images of parking spaces through cameras and uses traditional computer vision algorithms or deep learning models to identify the presence of vehicles and the degree of parking compliance. However, this technology is highly dependent on the integrity and clarity of image data. When the camera is blocked, or when there is inclement weather such as rain, fog, or snow, or insufficient lighting, the images are prone to problems such as blurring and missing parts, resulting in a significant decrease in the accuracy of parking space status judgment, especially making it difficult to accurately identify irregular parking behavior. Sensor detection technology (such as geomagnetic sensors, infrared sensors, etc.) determines whether there is a vehicle occupied by collecting changes in the magnetic field and infrared signals of the parking space area. However, sensor data has a single dimension and can only achieve a binary judgment of "car present / absent", which cannot distinguish between compliant parking and irregular parking (such as crossing the line, exceeding the parking space range, etc.), and is easily affected by environmental interference, leading to misjudgment.

[0003] To overcome the shortcomings of single-modal technologies, existing technologies have begun to explore multimodal data fusion detection, which combines multimodal information such as image data, sensor data, and environmental perception data for comprehensive judgment. However, current multimodal data fusion schemes have significant shortcomings: On the one hand, the weight allocation mechanism lacks flexibility, and most of them use fixed weights or empirical weights for data fusion. They cannot dynamically adjust the weight ratio of each submodal according to the reliability differences of modal data in different environmental scenarios (such as indoor parking lots, open-air parking spaces, busy road sections, remote road sections, etc.), resulting in poor environmental adaptability of the fusion results. When some modal data fails or its reliability decreases, the detection accuracy drops sharply. On the other hand, the dependence on image data has not been fundamentally solved. When no image data is available, it is difficult to achieve accurate three-level classification of "no car / standard parking / non-standard parking" by relying solely on other submodal data. It can only make a preliminary judgment on whether the parking space is occupied. When the image data is damaged (occluded, blurred, incomplete), the existing technology lacks an effective data completion mechanism. It cannot use the effective information of other submodal data to repair or complete the damaged image, causing the judgment of non-standard parking, which is highly dependent on image details, to completely fail.

[0004] Furthermore, most existing detection models based on multimodal data are not specifically optimized for parking space status classification scenarios, and they fail to utilize pre-trained models to uncover the intrinsic correlation between each submodal data and the parking space status labels (image data annotation results). This results in a lack of data support for submodal weight allocation, further reducing the stability and accuracy of the detection results. In summary, existing parking space status detection technologies suffer from poor robustness, insufficient environmental adaptability, low accuracy in judging irregular parking, and weak detection capabilities in scenarios without images, making it difficult to meet the actual needs of intelligent transportation systems for accurate and efficient parking space status detection. Summary of the Invention

[0005] This invention addresses the technical problems in existing technologies by providing an AI-based parking space status detection method based on multimodal data. The method includes: collecting historical multimodal data and concurrent historical image data of the target parking space; labeling the historical image data as no car, compliant parking, and non-compliant parking; constructing a training sample set by associating the category labels with the concurrent historical multimodal data; inputting the training sample set into a pre-trained model; and outputting the calibrated weights of each submodal data. The method also includes: collecting real-time multimodal data and environmental parameters of the current parking space; dynamically adjusting the calibrated weights based on the environmental parameters; performing a weighted calculation on the adjusted weights and the corresponding submodal data; and determining the parking space status based on the score range of the calculation result. If the determination result is non-compliant parking, the method calls up the image data and performs an integrity check on the image data. If the check fails, the missing areas of the image are filled in based on the calibrated weights, and the non-compliant parking status is verified by combining the filled image with the corrected image.

[0006] Furthermore, both historical and real-time multimodal data include at least two of the following: ultrasonic data, 24G radar data, photosensitive data, and geomagnetic data; environmental parameters include light intensity, raw weather data, parking space coordinates, and network time; historical image data and the image data retrieved are both panoramic images covering the entire parking space area, acquired through fixed or mobile acquisition devices.

[0007] Furthermore, historical multimodal data and concurrent historical image data of the target parking space are collected. The historical image data is labeled as no car, compliant parking, and non-compliant parking. The category labels are associated with the concurrent historical multimodal data to construct a training sample set. The training sample set is input into the pre-trained model, and the specific steps for outputting the calibrated weights of each submodal data are as follows: the training sample set is divided into a training set and a validation set; the pre-trained model weights are initialized; the historical multimodal data in the training set is input into the pre-trained model; the loss value for predicting the parking space status and category label is calculated; the weight parameters of each submodal data are adjusted through the backpropagation algorithm; after each training round, the model accuracy is evaluated using the validation set; the hyperparameters are adjusted based on the evaluation results; the iteration continues until the model converges; and the calibrated weights of each submodal data are output.

[0008] Furthermore, the specific details of dynamically adjusting the calibration weights based on environmental parameters include: acquiring network time and parking space coordinates, parsing the corresponding latitude and longitude information from the parking space coordinates, calling astronomical algorithms to combine the latitude and longitude information with the network time to calculate the local sunrise and sunset times, comparing the real-time network time with the sunrise and sunset times, and determining the environment as daytime if the real-time network time falls between sunrise and sunset; acquiring light intensity data, converting the light intensity into a luminance quantization value through a light sensor, and setting a correspondence between the luminance quantization value and the weight ratio of the photosensitive data in the daytime environment, with higher luminance quantization values ​​resulting in higher weights for the photosensitive data. The higher the percentage, the higher the calibration weight of geomagnetic data will be in nighttime environments. Raw weather data is acquired, and three indicators—visibility, precipitation intensity, and wind force—are extracted from it. Each indicator is assigned a score according to a preset scoring rule, and the scores are accumulated to obtain a quantitative value for the severity of the weather. A correspondence is established between the quantitative value for the severity of the weather and the weight adjustment gradient. For each gradient increase in the quantitative value, the weight percentage of 24G radar data increases by a preset percentage, while the weight percentage of ultrasonic data decreases by a corresponding percentage. When the quantitative value reaches a preset threshold, the weight percentage of 24G radar data is adjusted to be higher than that of ultrasonic data.

[0009] Furthermore, the adjusted weights and corresponding submodal data are weighted and calculated. The specific steps for determining the parking space status based on the score interval of the calculation result include: statistically analyzing the historical multimodal data and corresponding category labels in the training sample set to determine the first threshold for distinguishing between no cars and compliant parking, and the second threshold for distinguishing between compliant parking and non-compliant parking; multiplying the real-time multimodal data with the adjusted corresponding calibration weights respectively, and summing all the calculation results to obtain the weighted calculation result; if the weighted calculation result is less than the first threshold, the parking space is determined to be in a no-car state; if the weighted calculation result is between the first threshold and the second threshold, the parking space is determined to be in a compliant parking state; if the weighted calculation result is greater than the second threshold, the parking space is determined to be in a non-compliant parking state.

[0010] Further, the specific steps of invoking image data and performing integrity detection on the image data include: setting three image integrity detection indicators, namely the complete coverage rate of parking space lines, the clear rate of vehicle contours, and the non-occlusion rate of key areas; extracting the contour of the parking space line in the image through an edge detection algorithm, calculating the ratio of the length of the complete parking space line contour to the length of the standard parking space line to obtain the complete coverage rate of the parking space line; obtaining the vehicle contour through a contour extraction algorithm, and analyzing the sharpness of the contour edge and the degree of detail retention in combination with a clarity evaluation algorithm to obtain the clear rate of the vehicle contour; dividing the key area of the parking space through a region segmentation algorithm, and calculating the ratio of the area of the non-occluded area in the key area to the total area of the key area to obtain the non-occlusion rate of the key area; presetting the passing standards for each indicator, comparing the three extracted indicator parameters with the corresponding passing standards one by one. If all three indicator parameters meet the passing standards, it is determined that the image integrity detection is qualified; if any indicator parameter does not meet the passing standards, it is determined that the image integrity detection is unqualified.

[0011] Further, when the detection is unqualified, the specific steps of complementing the missing area of the image based on the calibration weight include: deploying multi-view acquisition devices around the parking space to collect multi-view images of the parking space from the horizontal direction, top view direction, and oblique direction respectively; extracting the characteristic information corresponding to the 24G radar data, ultrasonic data, photosensitive data, or geomagnetic data, and associating the extracted characteristic information with the calibration weights of each sub-modal data to obtain weighted characteristic information; performing region segmentation on the image with unqualified detection to identify the target areas of missing parking space lines, blurred vehicle contours, and occluded key areas; extracting the parking space positioning marker points in the multi-view images, recording the pixel coordinates of each marker point in different view images, establishing a unified coordinate system based on the pixel coordinates, and mapping each view image to this coordinate system. After completing the coordinate alignment of the multi-view images, they are stitched together to form an initial panoramic image; converting the initial panoramic image to the RGB color space, calculating the red, blue, and yellow color difference information of each pixel point, and assigning corresponding weights to the red color difference information, blue color difference information, and yellow color difference information according to the preset weight distribution rules. Among them, the weight of the red color difference information is used to strengthen the vehicle contour feature, the weight of the blue color difference information is used to strengthen the parking space line feature, and the weight of the yellow color difference information is used to strengthen the no-parking area identification feature, and the weighted three-color difference information is stitched together; retrieving the corresponding area features of the historical clear images of the same parking space in the same period, and combining the weighted characteristic information and the stitched color difference features to perform pixel-level filling on the target area to complete the complement of the missing area of the image.

[0012] Further, the multi-view acquisition device includes a fixed camera and an inspection robot. The deployment process is specifically as follows: installing the horizontal direction acquisition device on the side wall of the parking space, installing the top view direction acquisition device on the bracket directly above the parking space, and installing the oblique direction acquisition device on the column in the diagonal direction of the parking space. After installation, set the acquisition angles and shooting parameters of each device.

[0013] Furthermore, the pre-trained model is a deep learning model containing convolutional neural networks and fully connected layers. The loss value is calculated using the cross-entropy loss function. The criterion for model convergence is that the accuracy of the validation set remains stable above a preset accuracy threshold for a continuous preset number of rounds.

[0014] Furthermore, the verification of non-standard parking status using the completed image includes: extracting features from the completed image to identify four types of objects: parking lines, vehicle outlines, adjacent parking spaces, and no-parking zones; using parking lines as a reference, calculating the distance between each edge point of the vehicle outline and the corresponding parking line through image geometry to determine whether the vehicle exceeds the parking space limit; calculating the number of overlapping pixels between the vehicle outline and adjacent parking spaces and no-parking zones using pixel statistics methods, and obtaining the actual area of ​​the overlapping area by combining the conversion relationship between pixels and actual area; verifying whether the vehicle is non-standardly parked by combining the distance the vehicle exceeds the parking space limit with the actual area of ​​the overlapping area; if non-standard parking is confirmed, generating a warning message containing the parking space number, non-standard type, and image evidence, and pushing it to the parking lot management terminal.

[0015] The positive and progressive effects of this invention are as follows:

[0016] This method achieves dynamic adaptation of submodal weights through multimodal data fusion and pre-trained model optimization, effectively avoiding interference from environmental factors such as lighting and weather in single-modal scenarios. It can accurately determine the three parking space states without relying on image data, eliminating the dependence on image integrity in traditional detection methods. For damaged images, it combines calibrated weights, multi-view stitching, and color feature enhancement to achieve image restoration, ensuring the accuracy of non-standard parking verification. The overall solution improves the accuracy, stability, and environmental adaptability of parking space state detection, reduces manual inspection and error correction costs, and provides technical support for efficient management of intelligent parking lots. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the steps of a parking lot management method based on multimodal parking space detection according to the present invention. Detailed Implementation

[0018] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention.

[0019] Reference Figure 1A method for detecting parking space status based on multimodal data using AI includes: collecting historical multimodal data and historical image data of the target parking space; labeling the historical image data as no car, compliant parking, and non-compliant parking; constructing a training sample set by associating the category labels with the historical multimodal data; inputting the training sample set into a pre-trained model; and outputting the calibrated weights of each submodal data. The method also includes: collecting real-time multimodal data and environmental parameters of the current parking space; dynamically adjusting the calibrated weights based on the environmental parameters; performing weighted calculations on the adjusted weights and corresponding submodal data; and determining the parking space status based on the score range of the calculation results. If the determination result is non-compliant parking, the method calls up the image data and performs integrity checks on the image data. If the detection fails, the missing areas of the image are filled in based on the calibrated weights, and the non-compliant parking status is verified by combining the filled image with the complete image.

[0020] Furthermore, both historical and real-time multimodal data include at least two of the following: ultrasonic data, 24G radar data, photosensitive data, and geomagnetic data; environmental parameters include light intensity, raw weather data, parking space coordinates, and network time; historical image data and retrieved image data are both panoramic images covering the entire parking space area, acquired through fixed or mobile acquisition devices. In one example, a high-definition network camera can be used as the fixed acquisition device, and an inspection robot can be used as the mobile acquisition device. Through the collaborative acquisition of these two devices, the real-time monitoring of parking space status can be ensured, while avoiding blind spots in the coverage of a single acquisition method, thus ensuring the comprehensiveness of multimodal and image data.

[0021] Furthermore, historical multimodal data and concurrent historical image data of the target parking space are collected. The historical image data is labeled as no car, compliant parking, and non-compliant parking. The category labels are associated with the concurrent historical multimodal data to construct a training sample set. The training sample set is input into the pre-trained model, and the specific steps for outputting the calibrated weights of each submodal data are as follows: the training sample set is divided into a training set and a validation set; the pre-trained model weights are initialized; the historical multimodal data in the training set is input into the pre-trained model; the loss value for predicting the parking space status and category label is calculated; the weight parameters of each submodal data are adjusted through the backpropagation algorithm; after each training round, the model accuracy is evaluated using the validation set; the hyperparameters are adjusted based on the evaluation results; the iteration continues until the model converges; and the calibrated weights of each submodal data are output. In one example, the division of the training set and the validation set can be determined based on the uniformity of the data distribution, ensuring that the two datasets contain samples from different environments and parking space states. The convolutional neural network layer of the pre-trained model is responsible for extracting deep features from the multimodal data, while the fully connected layer maps the features to the weight values ​​of each submodality. The weight parameters are continuously optimized through the backpropagation algorithm, so that the calibrated weights can accurately match the correlation between different submodalities and parking space states, thereby improving the accuracy of subsequent detection.

[0022] Furthermore, the specific details of dynamically adjusting the calibration weights based on environmental parameters include: acquiring network time and parking space coordinates, parsing the corresponding latitude and longitude information from the parking space coordinates, calling astronomical algorithms to combine the latitude and longitude information with the network time to calculate the local sunrise and sunset times, comparing the real-time network time with the sunrise and sunset times, and determining the environment as daytime if the real-time network time falls between sunrise and sunset; acquiring light intensity data, converting the light intensity into a luminance quantization value through a light sensor, and setting a correspondence between the luminance quantization value and the weight ratio of the photosensitive data in the daytime environment, with higher luminance quantization values ​​resulting in higher weights for the photosensitive data. The higher the percentage of geomagnetic data, the higher its calibration weight will be in nighttime environments. Raw weather data is acquired, and three indicators—visibility, precipitation intensity, and wind speed—are extracted. Each indicator is assigned a score according to a preset scoring rule, and these scores are accumulated to obtain a quantified value for weather severity. A correspondence is established between the quantified value and the weight adjustment gradient. For each gradient increase in the quantified value, the weight of 24G radar data increases by a preset percentage, while the weight of ultrasonic data decreases accordingly. When the quantified value reaches a preset threshold, the weight of 24G radar data is adjusted to be higher than that of ultrasonic data. In one example, the astronomical algorithm can use a sunrise and sunset calculation method based on spherical trigonometry to ensure accurate time determination for different latitudes and longitudes and dates. The scoring rules for weather indicators can be set according to the degree of influence of each indicator on sensor detection; for example, lower visibility results in a higher score. Accumulation achieves a scientific quantification of severity, and gradient adjustment adapts the weight allocation to weather changes, ensuring the stability of detection under severe weather conditions.

[0023] Furthermore, the adjusted weights and corresponding submodal data are weighted and calculated. The specific steps for determining the parking space status based on the score range of the calculation result include: statistically analyzing historical multimodal data and corresponding category labels in the training sample set to determine a first threshold to distinguish between no cars and compliant parking, and a second threshold to distinguish between compliant and non-compliant parking; multiplying the real-time multimodal data with the adjusted corresponding calibrated weights, and summing all results to obtain the weighted calculation result; if the weighted calculation result is less than the first threshold, the parking space is determined to be no cars; if the weighted calculation result is between the first and second thresholds, the parking space is determined to be compliant parking; if the weighted calculation result is greater than the second threshold, the parking space is determined to be non-compliant parking. In one example, the threshold determination process can be achieved by statistically analyzing the distribution range of the weighted calculation results under different parking space statuses, ensuring that the first and second thresholds can clearly distinguish the three statuses and avoid ambiguity in the judgment; the weighted calculation achieves accurate judgment of the parking space status by highlighting the influence of high-reliability submodal data and reducing the weight ratio of low-reliability data under environmental interference.

[0024] Further, the specific steps of calling image data and performing integrity detection on the image data include: setting three image integrity detection indicators, namely the complete coverage rate of parking space lines, the clear rate of vehicle outlines, and the unobstructed rate of key areas; extracting the outline of the parking space line in the image through an edge detection algorithm, calculating the ratio of the length of the complete parking space line outline to the length of the standard parking space line to obtain the complete coverage rate of the parking space line; obtaining the vehicle outline through a contour extraction algorithm, and analyzing the sharpness of the contour edge and the degree of detail retention in combination with a clarity evaluation algorithm to obtain the clear rate of the vehicle outline; dividing the key area of the parking space through a region segmentation algorithm, and calculating the ratio of the area of the unobstructed area in the key area to the total area of the key area to obtain the unobstructed rate of the key area; presetting the qualified standard for each indicator, comparing the extracted three indicator parameters with the corresponding qualified standards one by one. If all three indicator parameters meet the qualified standards, it is determined that the image integrity detection is qualified; if any one of the indicator parameters does not meet the qualified standard, it is determined that the image integrity detection is unqualified. In one example, the edge detection algorithm can be the Canny algorithm, the contour extraction algorithm can adopt a contour detection function, and the clarity evaluation is realized by analyzing the change range of pixel gray values; the setting of the qualified standard is based on the minimum requirements of parking lot management for image recognition. The collaborative detection of the three indicators can comprehensively reflect whether the image meets the requirements for verifying irregular parking, and avoid missing image defects due to a single indicator.

[0025] Furthermore, the specific steps for filling in missing areas in the image based on calibration weights when the detection fails include: deploying multi-view acquisition devices around the parking space to acquire multi-view images of the parking space from horizontal, overhead, and diagonal directions; extracting feature information corresponding to 24G radar data, ultrasonic data, photosensitive data, or geomagnetic data, and associating the extracted feature information with the calibration weights of each sub-modal data to obtain weighted feature information; performing region segmentation on the images that fail the detection, identifying target areas with missing parking space lines, blurred vehicle outlines, or occlusion of key areas; extracting parking space positioning markers from the multi-view images, recording the pixel coordinates of each marker in different view images, establishing a unified coordinate system based on the pixel coordinates, and combining the images from each view. The image is mapped to this coordinate system, and after the coordinate alignment of the multi-view images is completed, they are stitched together to form an initial panoramic image. The initial panoramic image is converted into RGB color space, and the red, blue, and yellow difference information of each pixel is calculated. According to the preset weight allocation rules, corresponding weights are assigned to the red difference information, blue difference information, and yellow difference information respectively. The weight of the red difference information is used to enhance the vehicle outline features, the weight of the blue difference information is used to enhance the parking space line features, and the weight of the yellow difference information is used to enhance the no-parking zone sign features. The weighted three color difference information is then stitched together. The corresponding region features of the same parking space in the same period of clear historical images are retrieved. Combined with the weighted feature information and the stitched color difference features, pixel-level filling is performed on the target region to complete the missing areas of the image. In one example, the coordinate alignment of multi-view images is achieved through pixel coordinate mapping of unified marker points, ensuring that the initial panoramic image after stitching has no geometric deviation; the weight allocation of color difference information is based on the color feature differences of vehicles, parking lines, and no-parking areas, and the feature recognition is improved by strengthening the color boundary; during the pixel-level filling process, the combination of historical clear image features and weighted feature information enables the completed area to not only conform to the actual scene rules, but also be consistent with the features reflected by multimodal data, ensuring the authenticity and usability of the completed image.

[0026] Furthermore, the multi-view acquisition equipment includes fixed cameras and inspection robots. The deployment process is as follows: horizontal acquisition devices are installed on the side wall of the parking space, top-view acquisition devices are installed on the bracket directly above the parking space, and diagonal acquisition devices are installed on the diagonal pillars of the parking space. After installation, the acquisition angle and shooting parameters of each device are set. In one example, the acquisition angle is set to eliminate blind spots. The horizontal devices cover the entire side of the parking space, the top-view devices are vertically aligned with the center of the parking space, and the diagonal devices cover the diagonal area of ​​the parking space. The shooting parameters are dynamically adjusted according to the ambient lighting conditions to ensure that the images acquired under different environments have basic clarity, providing high-quality raw data for subsequent stitching and completion.

[0027] Furthermore, the pre-trained model is a deep learning model containing convolutional neural networks and fully connected layers. The loss value is calculated using the cross-entropy loss function. The convergence criterion for the model is that the accuracy on the validation set remains stable above a preset accuracy threshold for a consecutive preset number of rounds. In one example, the cross-entropy loss function provides a clear direction for weight adjustment by quantifying the difference between the prediction result and the class label. The requirement for stable accuracy across consecutive rounds is to avoid training termination due to accidental fluctuations, ensuring that the output labeled weights have good stability and reliability.

[0028] Furthermore, the verification of non-standard parking status using the completed image includes: feature extraction of the completed image to identify four types of objects: parking lines, vehicle outlines, adjacent parking spaces, and no-parking zones; using parking lines as a reference, image geometric calculations are used to obtain the distance between each edge point of the vehicle outline and the corresponding parking line to determine whether the vehicle exceeds the parking space limit; the number of overlapping pixels between the vehicle outline and adjacent parking spaces and no-parking zones is calculated using pixel statistics methods, and the actual area of ​​the overlapping area is obtained by combining the conversion relationship between pixels and actual area; the distance of the vehicle exceeding the parking space limit and the actual area of ​​the overlapping area are combined to verify whether the vehicle is non-standard parked; if non-standard parking is confirmed, a warning message containing the parking space number, non-standard type, and image evidence is generated and pushed to the parking lot management terminal. In one example, image geometric calculation is based on the mapping relationship between pixel coordinates and actual size, and the conversion between pixels and area is determined through pre-calibration; the warning message can be pushed through the pop-up window, SMS, or APP push function of the parking lot management system. The detailed information content can help managers quickly locate illegal parking spaces and take measures to deal with them, improving the efficiency of parking lot management.

[0029] The present invention has been described in detail above with reference to the accompanying drawings and embodiments. Those skilled in the art can make various modifications to the present invention based on the above description. Therefore, certain details in the embodiments should not be construed as limiting the present invention, and the scope of protection of the present invention shall be defined by the appended claims.

Claims

1. A method for detecting the status of AI parking spaces based on multi-modal data, characterized in that, include: Historical multimodal data and concurrent historical image data of the target parking space are collected. The historical image data is labeled as no car, compliant parking, and non-compliant parking. The category labels are associated with the concurrent historical multimodal data to construct a training sample set. The training sample set is input into the pre-trained model, and the calibration weights of each submodal data are output. Real-time multimodal data and environmental parameters of the current parking space are collected. The calibration weights are dynamically adjusted based on the environmental parameters. The adjusted weights are weighted and calculated with the corresponding submodal data. The parking space status is judged based on the score interval of the calculation result. If the judgment result is non-compliant parking, the image data is retrieved and the image data is checked for integrity. If the detection fails, the missing areas of the image are filled in based on the calibration weights. The non-compliant parking status is verified by combining the filled image.

2. The AI ​​parking space status detection method based on multi-modal data according to claim 1, characterized in that, Both historical and real-time multimodal data include at least two of the following: ultrasonic data, 24G radar data, photosensitive data, and geomagnetic data; environmental parameters include light intensity, raw weather data, parking space coordinates, and network time; historical image data and the image data retrieved are both panoramic images covering the entire parking space area, acquired through fixed or mobile acquisition devices.

3. The AI ​​parking space status detection method based on multi-modal data according to claim 2, characterized in that, Historical multimodal data and concurrent historical image data of the target parking space are collected. The historical image data is labeled as no car, compliant parking, and non-compliant parking. The category labels are associated with the concurrent historical multimodal data to construct a training sample set. The training sample set is input into the pre-trained model, and the specific steps for outputting the labeled weights of each submodal data are as follows: the training sample set is divided into a training set and a validation set; the pre-trained model weights are initialized; the historical multimodal data in the training set is input into the pre-trained model; the loss value for predicting the parking space status and category label is calculated; the weight parameters of each submodal data are adjusted through the backpropagation algorithm; after each training round, the model accuracy is evaluated using the validation set; the hyperparameters are adjusted based on the evaluation results; the iteration continues until the model converges; and the labeled weights of each submodal data are output.

4. The AI ​​parking space status detection method based on multi-modal data according to claim 2, characterized in that, The specific steps for dynamically adjusting calibration weights based on environmental parameters include: acquiring network time and parking space coordinates; parsing the corresponding latitude and longitude information from the parking space coordinates; using astronomical algorithms to calculate the local sunrise and sunset times by combining the latitude and longitude information with the network time; comparing the real-time network time with the sunrise and sunset times; if the real-time network time falls between sunrise and sunset, it is determined to be a daytime environment; otherwise, it is determined to be a nighttime environment; acquiring light intensity data; converting light intensity into luminance quantization values ​​using a light sensor; and setting a correspondence between luminance quantization values ​​and the weight ratio of photosensitive data in a daytime environment, with higher luminance quantization values ​​resulting in higher weight ratios for photosensitive data. The higher the preset ratio, the higher the calibration weight of geomagnetic data will be in nighttime environments. Obtain raw weather data and extract three indicators: visibility, precipitation intensity, and wind force. Assign scores to each indicator according to preset scoring rules and accumulate them to obtain a quantitative value of weather severity. Set a correspondence between the quantitative value of weather severity and the weight adjustment gradient. For each gradient increase in the quantitative value, the weight of 24G radar data will increase by a preset ratio, and the weight of ultrasonic data will decrease by a corresponding ratio. When the quantitative value reaches a preset threshold, the weight of 24G radar data will be adjusted to be higher than that of ultrasonic data.

5. The AI ​​parking space status detection method based on multi-modal data according to claim 1, characterized in that, The process of weighting the adjusted weights with the corresponding submodal data and determining the parking space status based on the score range of the calculation result includes: statistical analysis of historical multimodal data and corresponding category labels in the training sample set to determine a first threshold for distinguishing between no cars and compliant parking, and a second threshold for distinguishing between compliant and non-compliant parking; multiplying the real-time multimodal data with the adjusted corresponding calibrated weights and summing all the results to obtain the weighted calculation result; if the weighted calculation result is less than the first threshold, the parking space is determined to be no cars; if the weighted calculation result is between the first and second thresholds, the parking space is determined to be compliant parking; if the weighted calculation result is greater than the second threshold, the parking space is determined to be non-compliant parking.

6. The AI ​​parking space status detection method based on multi-modal data according to claim 1, characterized in that, The specific steps for calling image data and performing integrity checks include: setting three image integrity check indicators: parking line complete coverage rate, vehicle outline clarity rate, and key area unobstructed rate; extracting parking line outlines from the image using an edge detection algorithm, calculating the ratio of the complete parking line outline length to the standard parking line length to obtain the parking line complete coverage rate; obtaining vehicle outlines using an outline extraction algorithm, and analyzing the sharpness of the outline edges and the degree of detail retention using a clarity evaluation algorithm to obtain the vehicle outline clarity rate; dividing key areas of the parking space using a region segmentation algorithm, and calculating the ratio of the area of ​​the unobstructed area in the key area to the total area of ​​the key area to obtain the key area unobstructed rate; setting a pass / fail standard for each indicator, comparing the extracted three indicator parameters with the corresponding pass / fail standard one by one; if all three indicator parameters meet the pass / fail standard, the image integrity check is deemed qualified; if any indicator parameter fails to meet the pass / fail standard, the image integrity check is deemed unqualified.

7. The AI ​​parking space status detection method based on multi-modal data according to claim 2, characterized in that, When a parking space fails an inspection, the specific steps for filling in missing areas based on calibrated weights include: deploying multi-view acquisition devices around the parking space to acquire multi-view images of the parking space from horizontal, overhead, and diagonal directions; extracting feature information corresponding to 24G radar data, ultrasonic data, photosensitive data, or geomagnetic data, and associating the extracted feature information with the calibrated weights of each sub-modal data to obtain weighted feature information; performing region segmentation on the images that fail the inspection to identify target areas with missing parking lines, blurred vehicle outlines, or occlusion of key areas; extracting parking space positioning markers from the multi-view images, recording the pixel coordinates of each marker in different view images, establishing a unified coordinate system based on the pixel coordinates, and mapping the images from different view images. The images are projected into this coordinate system, and after coordinate alignment of the multi-view images, they are stitched together to form an initial panoramic image. The initial panoramic image is converted into an RGB color space, and the red, blue, and yellow difference information of each pixel is calculated. According to the preset weight allocation rules, corresponding weights are assigned to the red difference information, blue difference information, and yellow difference information respectively. The weight of the red difference information is used to enhance the vehicle outline features, the weight of the blue difference information is used to enhance the parking space line features, and the weight of the yellow difference information is used to enhance the no-parking zone sign features. The weighted three color difference information is then stitched together. The corresponding region features of the same parking space in the same period of clear historical images are retrieved. Combined with the weighted feature information and the stitched color difference features, pixel-level filling is performed on the target region to complete the missing areas of the image.

8. The AI ​​parking space status detection method based on multi-modal data according to claim 7, characterized in that, The multi-view acquisition equipment includes fixed cameras and inspection robots. The deployment process is as follows: the horizontal acquisition equipment is installed on the side wall of the parking space, the top-view acquisition equipment is installed on the bracket directly above the parking space, and the diagonal acquisition equipment is installed on the column diagonally opposite the parking space. After the installation is completed, the acquisition angle and shooting parameters of each device are set.

9. The AI ​​parking space status detection method based on multi-modal data according to claim 3, characterized in that, The pre-trained model is a deep learning model containing convolutional neural networks and fully connected layers. The loss value is calculated using the cross-entropy loss function. The criterion for model convergence is that the accuracy of the validation set remains stable above a preset accuracy threshold for a continuous preset number of rounds.

10. The AI ​​parking space status detection method based on multi-modal data according to claim 1, characterized in that, The specific steps for verifying non-standard parking status using the completed image include: extracting features from the completed image to identify four types of objects: parking lines, vehicle outlines, adjacent parking spaces, and no-parking zones; using parking lines as a reference, calculating the distance between each edge point of the vehicle outline and the corresponding parking line through image geometry to determine whether the vehicle exceeds the parking space limit; calculating the number of overlapping pixels between the vehicle outline and adjacent parking spaces and no-parking zones using pixel statistics methods, and obtaining the actual area of ​​the overlapping area by combining the conversion relationship between pixels and actual area; verifying whether the vehicle is non-standardly parked by combining the distance the vehicle exceeds the parking space limit with the actual area of ​​the overlapping area; if non-standard parking is confirmed, generating a warning message containing the parking space number, non-standard type, and image evidence, and pushing it to the parking lot management terminal.

Images