Parking lot and road sensing method and system, electronic equipment and medium

Abstract

The invention relates to the technical field of intelligent traffic, in particular to a parking lot and road sensing method and system, electronic equipment and a medium, and the method comprises the steps: firstly obtaining the image and video data of a parking lot and a road environment and obstacle distance data, carrying out the expansion of the image and video data, constructing a Faster-RCNN multi-target detection model, carrying out the feature extraction through a feature extraction network, and carrying out the recognition of the vehicle parking lot and road environment. According to the invention, a plurality of targets in a parking lot are detected and identified by using a region candidate network and generating a candidate region through the region candidate network, and meanwhile, obstacle distance data obtained by an ultrasonic sensor are used as assistance, and the defects of a visual sensor in complex environments such as low light and shielding are made up by measuring the distance between the ultrasonic sensor and the obstacle. The system can accurately sense the surrounding environment in various scenes, and the safety is further improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation technology, and in particular to a parking lot and road perception method, system, electronic equipment and medium. Background Art

[0002] As urban traffic continues to increase, parking management and road perception systems face increasingly severe challenges. Traditional parking management methods often rely on manual monitoring, which is inefficient and error-prone, and cannot meet the needs of modern transportation. In addition, existing road perception systems perform poorly in complex environments and have difficulty in accurately identifying and locating multiple targets. For example, CN115188181A discloses a multi-fusion road vehicle perception, navigation method and system, which specifically includes: the vehicle collects and uploads its own vehicle identification information and vehicle location information; and the vehicle with a perception sensor collects and uploads neighboring vehicle information; collects global vehicle driving information on the road; compares the vehicle identification information and vehicle location information to complete the vehicle identity recognition in the global vehicle driving information; and supplements and corrects the vehicle identity recognition results and vehicle location according to the neighboring vehicle information; CN118587871A discloses a road perception method, device, system, computing equipment and vehicle for vehicles, which includes: obtaining pre-processed radar data after pre-processing the radar raw data collected for road objects from the vehicle's millimeter-wave radar; obtaining driving information about the vehicle; based on the pre-processed radar data and driving information, using a deep learning model to determine the characteristic information of the road object; based on the characteristic information of the road object, determining the real-time congestion level of the vehicle on the road. Therefore, there is an urgent need for a new type of parking lot and road perception method, system, electronic equipment and medium to enhance the intelligent perception capabilities of parking lots and roads, optimize the parking management process, and thus improve overall traffic efficiency and safety. Summary of the invention

[0003] In order to solve the defects of existing parking lot and road perception systems, the present invention provides a parking lot and road perception method, system, electronic device and medium. By introducing the Faster-RCNN deep learning target detection algorithm and combining multiple sensor technologies, the intelligent perception capabilities of parking lots and roads are improved, the parking management and road monitoring processes are optimized, and the efficiency and safety of traffic management are further improved.

[0004] To achieve the above object, the present invention adopts the following technical solution: In a first aspect, the present invention provides a parking lot and road perception method, which specifically comprises the following steps: S1. Dataset collection: Obtain high-resolution images and video data of the parking lot and road environment through visual sensors (cameras), and obtain obstacle distance data through ultrasonic sensors; S2. Dataset expansion: The images and video data obtained by the visual sensor are used as the dataset, and the deep convolutional generative adversarial network (DCGAN) is used to expand the dataset; S3, model construction and training: build the Faster-RCNN multi-target detection model and train the model; S4, target detection: input the collected data into the trained Faster-RCNN multi-target detection model to obtain the parking lot and road detection results, and annotate the detection results on the original data; S5. System integration: Integrate the detection results with the existing traffic management system to provide real-time data support and optimize parking management and traffic flow control.

[0005] As a further technical solution of the present invention, the Faster-RCNN multi-target detection model constructed in step S3 uses the ResNet50 network as the main feature extraction network, extracts features from the data obtained by the visual sensor, and uses the FPN feature pyramid network for multi-scale feature fusion to obtain feature information of the parking lot and the road. Then, the RPN region generation network is used for target detection, and ROI Align is used instead of ROI pooling. The Soft NMS algorithm is used to reduce the misrepresentation of overlapping object detection frames, and the final result frame of the target detection is generated.

[0006] As a further technical solution of the present invention, DCN deformable convolution is used in the ResNet50 network to replace part of the ordinary convolution in ResNet50.

[0007] As a further technical solution of the present invention, the specific process of using the FCN feature pyramid network to perform multi-scale feature fusion is as follows: The features output by the last residual block layer of Conv2_x, Conv3_x, Conv4_x, and Conv5_x layers in the ResNet50 network are selected as the features of the input FPN feature pyramid network, denoted as {C2, C3, C4, C5}. The step sizes of these four feature layers relative to the original image are 4, 8, 16, and 32, respectively. The top-down and lateral connections are used to enlarge the small feature map on the top layer to the previous feature map by upsampling. Figure 1 That is, C5 is convolved with 1×1 to obtain P5, P5 is upsampled and added point by point with C4, where C4 is adjusted to be consistent with P5 in dimension by 1×1 convolution to obtain P4, and so on to obtain the fused multi-scale feature map {P2, P3, P4, P5}.

[0008] As a further technical solution of the present invention, the specific process of using the RPN region generation network for target detection is: The multi-scale feature map {P2, P3, P4, P5} is input into the RPN region generation network, which is traversed pixel by pixel and predicted for each pixel simultaneously. An initial candidate region called an anchor. For a The convolution feature map is obtained Anchors, sort all anchors by category probability, and select the top An anchor; Then, for each type of anchor selected, Soft NMS is uniformly performed to remove redundant candidate boxes, and finally the first m outputs are selected as the recommended candidate regions generated by the RPN region generation network. Soft NMS is expressed as follows: , in represents the score of the i-th candidate box, M and Represent the coordinates of the candidate box with the highest score and the coordinates of the i-th candidate box, respectively. represents the intersection-and-union ratio, is a predetermined threshold; for road information with irregular lane lines, the lane is detected as a series of points, and continuous points are merged to obtain a complete lane line.

[0009] As a further technical solution of the present invention, the process of training the Faster-RCNN multi-target detection model in step S3 is: inputting the data expanded in step S2 into the Faster-RCNN multi-target detection model for training, and continuously iterating and updating the model parameters according to the model detection results and the return loss until the detection accuracy of the model reaches the predetermined requirements.

[0010] As a further technical solution of the present invention, when the detection results are marked in step S4, parking lot signs, traffic signs, pedestrian and vehicle information are marked on the original image or video, and the obstacle position obtained by the ultrasonic sensor is marked. When the distance is too close, a reminder is issued to provide safety assistance.

[0011] In a second aspect, the present invention provides a parking lot and road perception system, comprising: The data set collection module is used to obtain high-resolution images and video data of the parking lot and road environment through visual sensors, and obtain obstacle distance data through ultrasonic sensors; The data set expansion module is used to expand the image and video data obtained by the visual sensor; Model building and training module, used to build and train the Faster-RCNN multi-target detection model; The target detection module is used to input the collected data into the trained Faster-RCNN multi-target detection model to obtain the parking lot and road detection results, and annotate the detection results on the original data; The system integration module is used to integrate the detection results with the existing traffic management system, provide real-time data support, and optimize parking management and traffic flow control.

[0012] In a third aspect, the present invention provides an electronic device comprising a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method described in the first aspect is completed.

[0013] In a fourth aspect, the present invention provides a computer-readable storage medium for storing computer instructions, wherein when the computer instructions are executed by a processor, the method described in the first aspect is performed.

[0014] Compared with the prior art, the present invention has the following beneficial effects: The present invention proposes a parking lot and road perception method, system, electronic device and medium to improve the accuracy of parking lot target recognition and road information perception. The innovation of the present invention is mainly reflected in five aspects: first, DCGAN is used to expand the training data set to help the target detection algorithm to continuously improve the effect; second, DCN deformable convolution is used to replace ordinary convolution to improve the detection and perception accuracy; third, ROI Align is used to replace ROI pooling to improve the detection accuracy of small targets in parking lots and roads such as traffic signs and long-distance road lines; fourth, FPN feature pyramid network is introduced to perform multi-scale feature fusion to make the multi-target detection network more accurate; fifth, Soft NMS is introduced to improve the performance of multi-target detection under occlusion conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their description are used to explain the present disclosure and do not constitute improper limitations on the present invention.

[0016] Figure 1 The network structure diagram of the Faster-RCNN multi-target detection model constructed for the present invention.

[0017] Figure 2 This is a flow chart of the parking lot and road perception method provided by the present invention.

[0018] Figure 3 This is a structural block diagram of the parking lot and road perception system provided by the present invention. DETAILED DESCRIPTION

[0019] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0020] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0021] It should be noted that the terms used herein are only for describing specific embodiments, and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that the terms "include" and "have" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0022] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0023] Embodiment 1: like Figure 2 As shown, this embodiment provides a parking lot and road perception method, which specifically includes the following steps: S1. Dataset collection: Obtain high-resolution images and video data of the parking lot and road environment through visual sensors, and obtain obstacle distance data through ultrasonic sensors. In this embodiment, a camera is used as a visual sensor to capture high-resolution real-time images and video data for identifying traffic signs, lane lines, pedestrians and other road features as the main input data to support subsequent multi-target detection. The obstacle distance measured by the ultrasonic sensor is used to evaluate the relative position and distance of obstacles, which is particularly suitable for complex scenes such as parking lots to improve safety and avoid collisions.

[0024] S2. Dataset expansion: The images and video data obtained by the visual sensor are used as the dataset, and the deep convolutional generative adversarial network (DCGAN) is used to expand the dataset to generate a diversified dataset to improve the generalization ability of the model in the subsequent training process. Data expansion includes image rotation, scaling, cropping, color transformation and other technologies to ensure that the model can adapt to different scenarios and conditions.

[0025] S3. Model construction and training: Construct a Faster-RCNN multi-target detection model and train the model. Specifically, the constructed Faster-RCNN multi-target detection model is as follows: Figure 1As shown in the figure, a residual neural network (ResNet50) with a depth of 50 is used to complete feature extraction, and DCN deformable convolution is used to replace some ordinary convolutions in ResNet50. Then, the FPN feature pyramid network is used to obtain multi-scale features. The features output by the last residual block layer of Conv2_x, Conv3_x, Conv4_x, and Conv5_x layers are selected as the features of the input FPN, denoted as {C2, C3, C4, C5}. The step sizes of these feature layers relative to the original image are 4, 8, 16, and 32, respectively. This is followed by a top-down and lateral connection process, where the small feature map on the top layer is enlarged to the previous feature map by upsampling. Figure 1 The size of the sample is fused by addition, which not only utilizes the semantic features of the top layer that are conducive to classification, but also utilizes the high-resolution information of the bottom layer that is conducive to positioning. For example, C5 is directly convolved with 1×1 to obtain P5; P5 is upsampled and added point by point with C4 (using 1×1 Conv to adjust the dimension to be consistent with P5) to obtain P4; and so on, a set of multi-scale feature maps {P2, P3, P4, P5} is obtained; then the region generation network (RPN) is used to generate the target candidate region. RPN uses the multi-scale feature map {P2, P3, P4, P5} as input, traverses it pixel by pixel, and predicts each pixel point at the same time. An initial candidate region called an anchor, then for a size of The convolution feature map of Anchors, sort all anchors by category probability, and select the top anchors; finally, Soft NMS is performed uniformly on each type of anchors selected to remove redundant candidate boxes ( ), select the first m outputs as the proposed candidate regions generated by the RPN network. Soft NMS is mathematically expressed as follows: , in represents the score of the i-th candidate box, M and Represent the coordinates of the candidate box with the highest score and the coordinates of the i-th candidate box, respectively. represents the intersection-and-union ratio, is a predetermined threshold; for irregular road information such as lane lines, the lane is detected as a series of points (minimal candidate boxes), and continuous points are merged to obtain a complete lane line.

[0026] The process of training the Faster-RCNN multi-target detection model is as follows: input the data expanded in step S2 into the Faster-RCNN multi-target detection model for training, and continuously iterate and update the model parameters according to the model detection results and return losses until the detection accuracy of the model reaches the predetermined requirements.

[0027] S4. Target detection: The collected data is input into the trained Faster-RCNN multi-target detection model to obtain the parking lot and road detection results, and the detection results are annotated on the original data. The parking lot logo, traffic signs, pedestrian and vehicle information are annotated on the original image or video. At the same time, the obstacle position obtained by the ultrasonic sensor is marked, and a reminder is issued when the distance is too close to provide safety assistance.

[0028] S5. System integration: Integrate the detection results with the existing traffic management system to provide real-time data support and optimize parking management and traffic flow control.

[0029] This embodiment first obtains high-resolution images, video data and obstacle distance data of the parking lot and road environment through visual sensors and ultrasonic sensors, and then processes the image data based on the Faster-RCNN algorithm, uses a feature extraction network (such as ResNet50) to extract features, and generates candidate regions through a region candidate network (RPN), ultimately achieving detection and recognition of multiple targets such as vehicles, pedestrians, and traffic signs in the parking lot. At the same time, the obstacle distance data obtained by the ultrasonic sensor is used as an auxiliary. By measuring the distance to the obstacle, it makes up for the shortcomings of the visual sensor in complex environments such as low light and occlusion, ensuring that the system can accurately perceive the surrounding environment in various scenarios and further improve safety.

[0030] Embodiment 2: like Figure 3 As shown, this embodiment provides a parking lot and road perception system, including: The data set collection module is used to obtain high-resolution images and video data of the parking lot and road environment through visual sensors, and obtain obstacle distance data through ultrasonic sensors; The data set expansion module is used to expand the image and video data obtained by the visual sensor; Model building and training module, used to build and train the Faster-RCNN multi-target detection model; The target detection module is used to input the collected data into the trained Faster-RCNN multi-target detection model to obtain the parking lot and road detection results, and annotate the detection results on the original data; The system integration module is used to integrate the detection results with the existing traffic management system, provide real-time data support, and optimize parking management and traffic flow control.

[0031] In further embodiments, there is also provided: An electronic device includes a memory and a processor, and is equipped with computer instructions running on the processor. When these instructions are executed by the processor, the method steps described in the embodiment can be accurately implemented. For the sake of simplicity, the detailed method content will not be repeated here.

[0032] It should be clear that the processor in this embodiment is not limited to the central processing unit CPU, it can also be a general-purpose processor core, a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic device PLD, a discrete gate circuit or transistor logic component, a discrete hardware module, etc. In addition, the general-purpose processor can be a microprocessor MPU, or any conventional computing core.

[0033] The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0034] A computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the method described in the embodiment is completed.

[0035] The method in the embodiment can be directly embodied as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0036] Those skilled in the art will appreciate that the units, i.e., algorithm steps, of the various examples described in conjunction with this embodiment can be implemented in electronic hardware or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0037] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0038] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A parking lot and road perception method, characterized in that: The specific steps include: S1. Dataset collection: Obtain high-resolution images and video data of the parking lot and road environment through visual sensors, and obtain obstacle distance data through ultrasonic sensors; S2. Dataset expansion: The images and video data obtained by the visual sensor are used as the dataset, and the deep convolutional generative adversarial network is used to expand the dataset; S3, model construction and training: build the Faster-RCNN multi-target detection model and train the model; S4, target detection: input the collected data into the trained Faster-RCNN multi-target detection model to obtain the parking lot and road detection results, and annotate the detection results on the original data; S5. System integration: Integrate the detection results with the existing traffic management system to provide real-time data support and optimize parking management and traffic flow control.

2. The parking lot and road perception method according to claim 1, characterized in that: The Faster-RCNN multi-target detection model constructed in step S3 uses the ResNet50 network as the main feature extraction network, uses the DCN deformable convolution to replace some ordinary convolutions in ResNet50, extracts features from the data obtained by the visual sensor, and uses the FPN feature pyramid network for multi-scale feature fusion to obtain the feature information of the parking lot and road. Then, the RPN region generation network is used for target detection, and ROI Align is used to replace ROI pooling. The Soft NMS algorithm is used to reduce the misrepresentation of overlapping object detection frames, and the final result frame of target detection is generated.

3. The parking lot and road perception method according to claim 2, characterized in that: The specific process of using the FCN feature pyramid network for multi-scale feature fusion is: The features output by the last residual block layer of the Conv2_x, Conv3_x, Conv4_x, and Conv5_x layers in the ResNet50 network are selected as the features of the input FPN feature pyramid network, denoted as {C2, C3, C4, C5}. The steps of these four feature layers relative to the original image are 4, 8, 16, and 32, respectively. The top-down and lateral connections are used to enlarge the small feature map of the top layer to the same size as the previous feature map by upsampling, and then fused by addition. That is, C5 is convolved with 1×1 to obtain P5, and P5 is upsampled and added point by point with C4. C4 uses 1×1 convolution to adjust the dimension to be consistent with P5 to obtain P4, and so on, to obtain the fused multi-scale feature map {P2, P3, P4, P5}.

4. The parking lot and road perception method according to claim 3, characterized in that: The specific process of using the RPN region generation network for target detection is: The multi-scale feature map {P2, P3, P4, P5} is input into the RPN region generation network, which is traversed pixel by pixel and predicted for each pixel simultaneously. An initial candidate region called an anchor. For a The convolution feature map is obtained anchors, sort all anchors by category probability, and select the top Anchors; Then, Soft NMS is uniformly performed on the anchors selected from each category to remove redundant candidate boxes. Finally, the first m outputs are selected as the recommended candidate regions generated by the RPN region generation network. Soft NMS is expressed as follows: ， in represents the score of the i-th candidate box, M and Represent the coordinates of the candidate box with the highest score and the coordinates of the i-th candidate box, respectively. represents the intersection-and-union ratio, is a predetermined threshold; for road information with irregular lane lines, the lane is detected as a series of points, and continuous points are merged to obtain a complete lane line.

5. The parking lot and road perception method according to claim 4, characterized in that: The process of training the Faster-RCNN multi-target detection model in step S3 is as follows: input the data expanded in step S2 into the Faster-RCNN multi-target detection model for training, and continuously iterate and update the model parameters according to the model detection results and the return loss until the detection accuracy of the model reaches the predetermined requirements.

6. The parking lot and road perception method according to claim 5, characterized in that: When marking the detection results in step S4, parking lot identification, traffic signs, pedestrian and vehicle information are marked on the original image or video, and the obstacle position obtained by the ultrasonic sensor is marked. When the distance is too close, a reminder is issued to provide safety assistance.

7. A parking lot and road perception system, characterized in that: The method according to any one of claims 1 to 6 is performed, comprising: The data set collection module is used to obtain high-resolution images and video data of the parking lot and road environment through visual sensors, and obtain obstacle distance data through ultrasonic sensors; The data set expansion module is used to expand the image and video data obtained by the visual sensor; Model building and training module, used to build and train the Faster-RCNN multi-target detection model; The target detection module is used to input the collected data into the trained Faster-RCNN multi-target detection model to obtain the parking lot and road detection results, and annotate the detection results on the original data; The system integration module is used to integrate the detection results with the existing traffic management system, provide real-time data support, and optimize parking management and traffic flow control.

8. An electronic device, characterized in that: The invention comprises a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method according to any one of claims 1 to 6 is completed.

9. A computer-readable storage medium, characterized in that: Used to store computer instructions, which, when executed by a processor, complete the method as claimed in any one of claims 1 to 6.

Images