Rearview mirror withdrawing detection and early warning system and method for vehicle entering intelligent stereo garage

Abstract

The invention discloses a rearview mirror withdrawing detection and early warning system and method for a garage entering vehicle of an intelligent stereo garage, and the method comprises the steps: completing the positioning operation of a pre-garage entering vehicle at an entrance of the intelligent stereo garage; a camera at the top of a garage entrance shoots a top view of a vehicle to be put into a garage, the top view is input into the FPN improved Faster RCNN model, and the position of a rearview mirror is detected; an improved fast two-dimensional OTSU threshold segmentation algorithm is combined with a pixel image processing technology to judge whether the rearview mirror is retracted or not, and if the judgment result is that the rearview mirror is in a retracted state, processing is not carried out; if the judgment result is that the rearview mirror is not retracted, whether the vehicle is in the suitable parking range or not is judged again, if the vehicle exceeds the suitable parking range, an alarm is given to remind a vehicle owner to retract the rearview mirror, and if not, processing is not conducted. The intelligent degree of the stereo garage is improved, and meanwhile the possibility that vehicles are damaged is reduced.

Description

Technical Field

[0001] The invention relates to the field of intelligent stereoscopic garages, and in particular to detecting whether a rearview mirror of a vehicle to be stored in a garage has been retracted and giving an alarm when necessary. Background Art

[0002] With the rapid development of my country's economy, the rapid popularization of motor vehicles in families has become a reality. Various cars, SUVs, MPVs and other models have entered every household. However, with the increase in the number of motor vehicles, the problem of insufficient parking spaces has become more urgent. To solve this problem, many different types of intelligent stereo garages have emerged on the market.

[0003] In order to make full use of limited space resources and maximize space utilization, as the widest component on the vehicle, most smart stereo garages often default to the retracted state of the rearview mirror when designing parking space specifications to reduce the distance between adjacent vehicles and save parking space. However, in real life, it often happens that car owners forget to retract the rearview mirror after parking, and the unretracted rearview mirror will inevitably cause scratches during the vehicle storage and retrieval process, resulting in unnecessary economic losses. Considering the wide variety of existing models, many manufacturers have launched models with smaller specifications than before in order to cater to the market. For this type of vehicle, conventional standard parking spaces can still safely meet parking needs even if the vehicle rearview mirror is not retracted. In this case, blindly requiring car owners to retract the rearview mirror when parking is obviously not user-friendly and unnecessary.

[0004] Therefore, how to effectively detect whether the vehicle's rearview mirror is retracted and issue an alarm when necessary, while saving space and avoiding damage to the vehicle, has become a problem that needs to be solved urgently. Summary of the invention

[0005] In view of the shortcomings in the prior art, the present invention provides a rearview mirror retraction detection and early warning system and method for vehicles entering an intelligent three-dimensional parking garage. The system detects the status of the rearview mirrors of the vehicles to be entered into the garage. If the rearview mirrors are not retracted, it determines whether the vehicle is within the suitable parking range. If not, an alarm is issued to remind the owner to retract the rearview mirrors in time to prevent unnecessary scratches.

[0006] The present invention achieves the above technical objectives through the following technical means.

[0007] A method for detecting and warning the retraction of rearview mirrors of vehicles entering an intelligent three-dimensional garage:

[0008] At the entrance of the intelligent three-dimensional garage, the vehicles to be stored in the garage complete the positioning operation;

[0009] The camera on the top of the garage entrance takes a bird's-eye view of the vehicle to be stored in the garage, which is input into the FPN-modified Faster RCNN model to detect the position of the rearview mirror.

[0010] The top view is input into the Faster RCNN model improved by FPN to detect the position of the rearview mirror, specifically including: inputting the top view into FPN for feature extraction to obtain 4-level feature maps fused by shallow features and deep features; inputting each level of feature map into RPN to preliminarily obtain the region of interest; inputting the region of interest and the feature map into ROIHead to locate the final target; performing convolution pooling operation on the output result of ROIHead, and then using two parallel convolution layers for classification and regression to obtain the final target detection, that is, the position of the rearview mirror;

[0011] The improved fast two-dimensional OTSU threshold segmentation algorithm is combined with pixel image processing technology to determine whether the rearview mirror is retracted. Specifically, the average grayscale value of the neighborhood of each pixel in the block diagram rearM corresponding to the rearview mirror position is calculated to construct a binary group (i, j). The probability of the binary group (i, j) occurring is recorded as Among them, i and j represent the grayscale value of the pixel in rearM and the grayscale mean of its neighborhood, r ij is the frequency of occurrence of the binary group (i, j), M×N is the size of rearM; introduce the weight matrix W(i, j) to construct the probability distribution By probability distribution Construct an improved two-dimensional OTSU threshold segmentation algorithm objective function tr(σ rM ); solve the binary pair (s, t) so that the tr (σ rM ) is the largest, and the optimal threshold is obtained Where s and t represent the thresholds of image background and target respectively; according to the optimal threshold Binarize rearM, add the pixels of the binary image by row and column respectively, and get two histograms at different angles. The range of the pixel number mutation part of the two histograms is d0 and d1, and compare: if d0>d1, the rearview mirror has been retracted, otherwise it is considered that the rearview mirror has not been retracted. Then judge the distance between the two rearview mirrors and the specification adaptability of the parking space. If it does not meet the requirements, an alarm will be issued to remind the owner to retract the rearview mirror.

[0012] Furthermore, the top view is input into FPN for feature extraction to obtain 4-level feature maps fused by shallow features and deep features, specifically: using ResNet-50 as the backbone, the image is first convolved to extract deep features of the image to obtain 5 stages: {C1, C2, C3, C4, C5}, and the last four stages are taken as bottom-up pyramid layers; then a 1×1 convolution is performed on each level of the pyramid layer as a skip connection; then the top layer C5 of the pyramid layer is upsampled with a ratio of 2, and the corresponding skip connection is added. The above operation is repeated for C4, C3 and C2 to obtain 4-level feature maps fused by shallow features and deep features: {P2, P3, P4, P5}; finally, a 3×3 convolution is performed on each level of the feature map.

[0013] Furthermore, the region of interest and the feature map are input into ROIHead to locate the final target. Specifically, the regions of interest of different scales are assigned to the feature maps P of the corresponding levels according to the following formula: k′ superior:

[0014]

[0015] Among them, the level k'=2,3,4,5, w, h are the width and height of the region of interest respectively, and k0 is a definable constant.

[0016] Furthermore, the objective function of the two-dimensional OTSU threshold segmentation algorithm is constructed and improved, specifically:

[0017] The probabilities of the background and the target corresponding to each other are denoted as P0 and P1 respectively, and the corresponding grayscale means are denoted as μ0 and μ1; and:

[0018]

[0019]

[0020] Construct the discreteness matrix σ rM :

[0021]

[0022] The trace of the discreteness matrix is ​​used as the discreteness measure, that is, the objective function:

[0023]

[0024] Among them, μ T =(μ Ti , μ Tj ) is the total grayscale mean vector of the image, where P kis the probability that the background corresponds to the target, and L represents the grayscale level of the average image of the neighborhood composed of the average grayscale value.

[0025] Furthermore, the solution of the binary pair (s, t) is such that tr(σ rM ) is the largest, and the optimal threshold is obtained Specifically:

[0026] Initialize parameters s0, t0 and line segments f0, f1; s0, t0 are the initial values ​​of s, t respectively, and f0, f1 are the upper and lower boundaries parallel to the diagonal of the two-dimensional OTSU projection image respectively;

[0027] Keeping s0 unchanged, calculate the line segment between f0 and f1 so that tr(σ rM ) takes the maximum value of t, denoted as t1;

[0028] Keeping t1 unchanged, calculate the line segment between f0 and f1 so that tr(σ rM ) takes the maximum value of s, recorded as s1;

[0029] Repeat the above steps until a solution s is obtained. n or n If it has already appeared in the solution process, or the number of iterations reaches the upper limit, the loop is exited and the optimal threshold is obtained.

[0030] Furthermore, the compatibility of the distance between the two rearview mirrors with the specifications of the parking spaces is judged by calculating the number of pixels occupied by the horizontal tracks and the vehicle rearview mirrors in the top view of the pre-parked vehicle, and calculating the distance between the two rearview mirrors of the vehicle when the rearview mirrors are not retracted in proportion based on the actual distance between the horizontal tracks, and comparing the width suitable for parking in the smart three-dimensional garage.

[0031] Furthermore, at the entrance of the intelligent three-dimensional garage, the pre-parked vehicle completes the positioning operation, specifically: when the vehicle enters the garage, the front of the vehicle reaches the position of the infrared device 1, and the indicator light turns from red to green under the induction of the infrared device, indicating that the positioning is completed.

[0032] Furthermore, the FPN-improved Faster RCNN model uses FPN to replace the original feature extraction part of the Faster RCNN model.

[0033] Furthermore, the RPN sets a 3×3 convolution and two 1×1 convolutions, the former is used to concentrate features, and the latter two are used to generate the initially screened regions of interest.

[0034] A rearview mirror retraction detection and early warning system for vehicles entering an intelligent three-dimensional garage, comprising:

[0035] The pre-warehouse vehicle positioning system uses infrared devices to locate the pre-warehouse vehicle to ensure that the vehicle position conditions meet the requirements of rearview mirror detection;

[0036] The rearview mirror detection system for pre-parked vehicles uses a camera to capture a bird's-eye view of the pre-parked vehicle and inputs the image into a pre-trained FPN-modified Faster RCNN model to detect and locate the position of the rearview mirror.

[0037] The rearview mirror status judgment and warning system judges the status of the rearview mirror according to the output results of the rearview mirror detection system of the pre-parked vehicle, and determines whether to issue an alarm based on the comprehensive judgment results to remind the vehicle owner.

[0038] The beneficial effects of the present invention are:

[0039] (1) Compared with the traditional intelligent stereo garage, the fully automated processing flow of the present invention reduces manual participation, improves the intelligence level of the stereo garage, and reduces the possibility of vehicle damage.

[0040] (2) The Faster RCNN model improved by FPN in the present invention can further extract multi-scale features, has higher accuracy and stability for small target detection, and is more suitable for rearview mirror detection scenarios.

[0041] (3) The present invention utilizes a fast two-dimensional OTSU threshold segmentation algorithm to more accurately separate the main body of the rearview mirror, and combines image processing technology to further quickly determine the state of the rearview mirror; after determining the state of the rearview mirror, the degree of specification adaptation between the vehicle and the parking space is further determined, avoiding blind requests to retract the rearview mirror, which is more user-friendly. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A flow chart of rearview mirror retraction detection and early warning for vehicles entering the intelligent three-dimensional garage according to the present invention;

[0043] Figure 2 This is a schematic diagram of positioning the pre-warehouse vehicle according to the present invention;

[0044] FIG3( a ) is a schematic diagram of the garage entrance layout of the present invention;

[0045] FIG3( b ) is a schematic diagram of the vehicle pre-entry into storage according to the present invention;

[0046] Figure 4 This is a flow chart of rearview mirror detection according to the present invention;

[0047] Figure 5 This is a flow chart of rearview mirror status determination and alarm according to the present invention;

[0048] Figure 6It is the two-dimensional OTSU projection diagram of the present invention;

[0049] In the figure, 1-infrared device, 2-camera, 3-horizontal track, 4-conveyor. DETAILED DESCRIPTION

[0050] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments, but the protection scope of the present invention is not limited thereto.

[0051] The present invention provides a rearview mirror retraction detection and early warning system for vehicles entering an intelligent three-dimensional garage, comprising:

[0052] Pre-warehouse vehicle positioning system, which uses infrared devices to locate the pre-warehouse vehicle to ensure that the vehicle position conditions meet the requirements of subsequent rearview mirror detection;

[0053] The rearview mirror detection system for pre-parked vehicles uses a camera to capture a bird's-eye view of the pre-parked vehicle and automatically and intelligently recognizes the image to detect and locate the position of the rearview mirror;

[0054] The rearview mirror status judgment and warning system judges the status of the rearview mirror according to the output results of the rearview mirror detection system of the pre-parked vehicle, and determines whether to issue an alarm based on the comprehensive judgment results to remind the owner.

[0055] The specific operation is as follows: the owner drives the vehicle into the entrance of the intelligent stereo garage and stops on the horizontal track 3. After the infrared device of the pre-warehouse vehicle positioning system determines the vehicle position that meets the subsequent detection, the owner can get off the vehicle and leave; then the pre-warehouse vehicle rearview mirror detection system uses the camera on the top of the garage to take a bird's-eye view of the pre-warehouse vehicle, and inputs the image into the pre-trained FPN improved Faster RCNN model, and the FPN extracts features from the image, and the Faster RCNN detects and locates the rearview mirror, and outputs the position result of the target rearview mirror; then, the rearview mirror detection result is input into the rearview mirror state judgment and early warning system, and the detection result image is binarized according to the category separability criterion, and the rearview mirror state is determined according to the pixel distribution state of the binarized image; if the judgment result is that the rearview mirror is in the retracted state, no processing is performed; if the judgment result is that the rearview mirror is not retracted, it is judged again whether the vehicle is in the suitable parking range, and if it is out of the range, an alarm is issued to remind the owner to retract the rearview mirror, otherwise no processing is performed.

[0056] like Figure 1As shown in the figure, when a vehicle enters the intelligent stereo garage, the pre-parked vehicle is first positioned to ensure that the vehicle position meets the requirements of the rearview mirror detection; then the camera takes a bird's-eye view of the pre-parked vehicle, and inputs the target detection model to detect and locate the rearview mirror. The rearview mirror status is determined based on the detection result diagram, the status of the rearview mirror is confirmed, and different processing methods are taken accordingly.

[0057] The entrance layout of the intelligent three-dimensional garage is as follows Figure 2 , 3(a) As shown in Figure 3(b), it includes an infrared device 1 (including an indicator light), a camera 2, a horizontal track 3 along the direction of vehicle entry, and a carrier 4. The infrared device 1 is installed 60 to 70 cm in front of the horizontal track 3 and 130 to 140 cm from the entry door. The camera 2 is installed at the top of the garage entrance and is located directly above the middle of the horizontal track 3. The carrier 4 is located between the horizontal tracks 3 on both sides, and the two sides of the carrier 4 are respectively embedded with the horizontal tracks 3. When the vehicle enters along the horizontal track 3, the front of the vehicle reaches the position of the infrared device 1, and the infrared device 1 is blocked to meet the needs of the top camera 2 to shoot. At this time, the indicator light turns from red to green under the induction of the infrared device 1, indicating that the positioning needs have been completed. When all pre-entry inspections (including the camera taking images for rearview mirror retraction detection) are completed, the vehicle will be transported by the carrier 4 and sent to the parking area along the horizontal track 3 to complete the subsequent vehicle entry process.

[0058] Among them, the infrared device 1, the camera 2 and the carrier 4 all communicate with the controller. In the controller, it is detected whether the rearview mirror of the pre-parked vehicle is retracted, and an alarm is issued to the owner according to the detection result, or no action is taken; in the rearview mirror retraction detection part, target detection and state determination are performed based on the image taken by the camera on the top of the garage, so as to realize the main functions of the system; the details are as follows:

[0059] (1) Rearview mirror position detection (such as Figure 4 (shown)

[0060] Step 1: Camera 2 at the top of the garage entrance takes a bird's-eye view of the pre-parked vehicle as input to the subsequent target detection model (i.e., the FPN-improved Faster RCNN model, specifically, FPN is used to replace the original feature extraction part of the Faster RCNN model).

[0061] Step 2: Input the top view image into the FPN-modified Faster RCNN model to detect the position of the rearview mirror.

[0062] i. Input the image into FPN for feature extraction

[0063] Using ResNet-50 as the backbone, we first perform convolution on the image to extract deep features of the image, and obtain 5 stages: {C1, C2, C3, C4, C5}. The last four stages are taken as bottom-up pyramid layers. Then, we perform 1×1 convolution on each level of the pyramid layer as a skip connection for subsequent feature fusion. Then, we take the top layer C5 of the pyramid layer and perform upsampling with a ratio of 2, and add the corresponding skip connection. The above operation is repeated for C4, C3, and C2 to obtain a feature map of 4 levels of shallow features and deep features from top to bottom: {P2, P3, P4, P5}. Finally, we perform 3×3 convolution on each level of feature map to reduce the aliasing effect of upsampling.

[0064] ii. Input each level of feature map into RPN to initially obtain the region of interest roi

[0065] A 3×3 convolution and two 1×1 convolutions are set as the structure of RPN. The former is used to further concentrate the features, and the latter two are used to generate the initially screened region of interest (ROI).

[0066] iii. Input the region of interest roi and feature map into ROIHead to locate the final target

[0067] The regions of interest roi of different scales obtained by RPN are assigned to the feature maps P of the corresponding levels according to the following formula k′ (Level k'=2,3,4,5) on:

[0068]

[0069] Among them, w and h are the width and height of the region of interest roi respectively, and the definable constant k0 is set to 4.

[0070] iv. Classification and regression

[0071] The output of ROIHead is subjected to convolution and pooling operations, and then two parallel convolutional layers are used for classification and regression to obtain the final target detection (i.e., the position of the rearview mirror).

[0072] Step 3: According to the block diagram coordinates corresponding to the rearview mirror position obtained previously, separate the rearview mirror part for subsequent rearview mirror status detection.

[0073] (2) Rearview mirror status determination (such as Figure 5 (shown)

[0074] In the rearview mirror state determination part, an improved fast two-dimensional OTSU threshold segmentation algorithm is designed based on the results of the rearview mirror detection part, combined with pixel image processing technology to determine whether the rearview mirror is retracted. The specific description is as follows:

[0075] Input: Block diagram corresponding to the rearview mirror position rearM

[0076] Output: rearview mirror state

[0077] Step 1: Input image rearM, calculate the neighborhood average grayscale value of each pixel of rearM, and construct a tuple.

[0078] Calculate the average gray value of the 3×3 neighborhood of each pixel in the image. The gray level of the neighborhood average image composed of the average gray value is the same as that of the original image, which is set to L.

[0079] Construct a binary group (i, j), where i and j represent the grayscale value of the pixel in the image and the grayscale mean of its neighborhood respectively; the probability of the binary group (i, j) occurring is where r ij is the frequency of occurrence of the binary group (i, j), and M×N is the size of the image rearM.

[0080] In order to improve the image segmentation effect of the algorithm, the weight matrix W(i, j) is introduced to construct a new probability distribution By assigning a larger weight to the edge area of ​​the rearview mirror, the algorithm's sensitivity to important image features is improved, while the impact of noise on algorithm recognition is reduced.

[0081] Step 2: Construct an improved two-dimensional OTSU threshold segmentation algorithm objective function.

[0082] like Figure 5 As shown, the diagonal areas 1 and 2 represent the image background and target respectively. When the thresholds of the image background and target are (s, t), the probabilities of the background and target corresponding to each other are recorded as P0 and P1 respectively, and the corresponding grayscale means are recorded as μ0 and μ1; where:

[0083]

[0084] Construct the dispersion matrix:

[0085]

[0086] The trace of the discreteness matrix is ​​used as the discreteness measure, that is, the objective function:

[0087]

[0088] Among them, μ T =(μ Ti, μ Tj ) is the total grayscale mean vector of the image, where P k is the probability corresponding to the background and the target.

[0089] Step3: Solve the binary tuple (s, t) to maximize tr(σ rM ), and obtain the optimal threshold

[0090] Convert the solution of the binary tuple into the solution of a single variable, and quickly solve the above objective function by the bisection method. Regarding (s, t) as continuous variables, then (s, t) constitutes a surface. Utilize the extreme value characteristics in the s and t directions, and use the bisection method to gradually approach the optimal solution. At the same time, limit the parameter solution range within a certain range of the diagonal to accelerate the solution. Denote the range boundaries as f:

[0091] Initialize the parameters s0, t0 and the line segments f0, f1; where s0 and t0 are the initial values of s and t respectively, and f0 and f1 are the upper and lower boundaries parallel to the diagonal;

[0092] Keep s0 unchanged, and calculate the t that maximizes tr(σ rM ) on the line segment between f0 and f1, denoted as t1;

[0093] Keep t1 unchanged, and calculate the s that maximizes tr(σ rM ) on the line segment between f0 and f1, denoted as s1;

[0094] Repeat the above operations until a certain s n or t n has already appeared in the solution process, or the number of iteration steps reaches the upper limit, then exit the loop. At this time, the optimal threshold

[0095] Step4: Binarize the image and judge the state of the rearview mirror using the pixel distribution

[0096] According to the optimal threshold perform binarization on the original image rearM. The part less than or equal to the threshold is taken as 0, and the part greater than the threshold is taken as 1.

[0097] Add the pixels of the binarized image row by row and column by column to obtain two histograms T0 and T1 at different angles. Denote the ranges d0 and d1 of the pixel number mutation parts of the two histograms, and compare: if d0 > d1, the rearview mirror has been retracted, denoted as state0; if d0 < d1, the rearview mirror has not been retracted, denoted as state1.

[0098] Step5: Output the state of the rearview mirror

[0099] Different operations are performed according to the status of the rearview mirror. If the rearview mirror is retracted, the warning system does not work; if the rearview mirror is not retracted, the compatibility of the specifications between the vehicle and the parking space is judged.

[0100] The compatibility judgment between the vehicle and the parking space is also based on the image taken by the front camera. The number of pixels occupied by the horizontal track and the vehicle's rearview mirror on the image is calculated, and combined with the actual distance between the horizontal tracks, the distance between the two rearview mirrors of the vehicle when the rearview mirror is not retracted is calculated proportionally, and compared with the width suitable for parking in the smart three-dimensional garage. If it meets the requirements, no processing will be done, otherwise an alarm will be issued to remind the owner to retract the rearview mirror.

[0101] The embodiments are preferred implementations of the present invention, but the present invention is not limited to the above-mentioned implementations. Any obvious improvements, substitutions or modifications that can be made by those skilled in the art without departing from the essential content of the present invention belong to the protection scope of the present invention.

Claims

1. A method for detecting and warning the retraction of rearview mirrors of vehicles entering an intelligent three-dimensional garage, characterized in that: At the entrance of the intelligent three-dimensional garage, the vehicles to be stored in the garage complete the positioning operation; The camera on the top of the garage entrance takes a bird's-eye view of the vehicle to be stored in the garage, which is input into the FPN-modified Faster RCNN model to detect the position of the rearview mirror. The top view is input into the Faster RCNN model improved by FPN to detect the position of the rearview mirror, specifically including: inputting the top view into FPN for feature extraction to obtain 4-level feature maps fused by shallow features and deep features; inputting each level of feature map into RPN to preliminarily obtain the region of interest; inputting the region of interest and the feature map into ROIHead to locate the final target; performing convolution pooling operation on the output result of ROIHead, and then using two parallel convolution layers for classification and regression to obtain the final target detection, that is, the position of the rearview mirror; The improved fast two-dimensional OTSU threshold segmentation algorithm is combined with pixel image processing technology to determine whether the rearview mirror is retracted. Specifically, the average grayscale value of the neighborhood of each pixel in the block diagram rearM corresponding to the rearview mirror position is calculated to construct a binary group (i, j). The probability of the binary group (i, j) occurring is recorded as Among them, i and j represent the grayscale value of the pixel in rearM and the grayscale mean of its neighborhood, r ij is the frequency of occurrence of the binary group (i, j), M×N is the size of rearM; introduce the weight matrix W(i, j) to construct the probability distribution By probability distribution Construct an improved two-dimensional OTSU threshold segmentation algorithm objective function tr(σ rM ); solve the binary pair (s, t) so that the tr (σ rM ) is the largest, and the optimal threshold is obtained Where s and t represent the thresholds of image background and target respectively; according to the optimal threshold Binarize rearM, add the pixels of the binary image by row and column respectively, and get two histograms at different angles. The range of the pixel number mutation part of the two histograms is d0 and d1, and compare: if d0>d1, the rearview mirror has been retracted, otherwise it is considered that the rearview mirror has not been retracted. Then judge the distance between the two rearview mirrors and the specification adaptability of the parking space. If it does not meet the requirements, an alarm will be issued to remind the owner to retract the rearview mirror.

2. The rearview mirror retraction detection and warning method according to claim 1, characterized in that: The top view is input into FPN for feature extraction to obtain 4-level feature maps fused by shallow features and deep features, specifically: using ResNet-50 as backbone, the image is first convolved to extract deep features of the image to obtain 5 stages: {C1, C2, C3, C4, C5}, and the last four stages are taken as bottom-up pyramid layers; then 1×1 convolution is performed on each level of the pyramid layer as skipconnection; then the top layer C5 of the pyramid layer is upsampled with a ratio of 2, and the corresponding skipconnection is added, and the above operation is repeated for C4, C3 and C2 to obtain 4-level feature maps fused by shallow features and deep features: {P2, P3, P4, P5}; finally, 3×3 convolution is performed on each level of the feature map.

3. The rearview mirror retraction detection and warning method according to claim 2, characterized in that: The region of interest and the feature map are input into ROIHead to locate the final target. Specifically, the regions of interest of different scales are assigned to the feature maps P of the corresponding levels according to the following formula: k′ superior: Among them, the level k'=2,3,4,5, w, h are the width and height of the region of interest respectively, and k0 is a definable constant.

4. The rearview mirror retraction detection and warning method according to claim 1, characterized in that: The objective function of the two-dimensional OTSU threshold segmentation algorithm is constructed and improved, specifically: The probabilities of the background and the target corresponding to each other are denoted as P0 and P1 respectively, and the corresponding grayscale means are denoted as μ0 and μ1; and: Construct the discreteness matrix σ rM : The trace of the discreteness matrix is ​​used as the discreteness measure, that is, the objective function: Among them, μ T =(μ Ti , μ Tj ) is the total grayscale mean vector of the image, where P k is the probability that the background corresponds to the target, and L represents the grayscale level of the average image of the neighborhood composed of the average grayscale value.

5. The rearview mirror retraction detection and warning method according to claim 1, characterized in that: The solution of the binary pair (s, t) is to make the tr (σ rM ) is the largest, and the optimal threshold is obtained Specifically: Initialize parameters s0, t0 and line segments f0, f1; Among them, s0 and t0 are the initial values ​​of s and t, respectively, and f0 and f1 are the upper and lower boundaries parallel to the diagonal of the two-dimensional OTSU projection graph; Keeping s0 unchanged, calculate the line segment between f0 and f1 so that tr(σ rM ) takes the maximum value of t, denoted as t1; Keeping t1 unchanged, calculate the line segment between f0 and f1 so that tr(σ rM ) takes the maximum value of s, recorded as s1; Repeat the above steps until a solution s is obtained. n or n If it has already appeared in the solution process, or the number of iterations reaches the upper limit, the loop is exited and the optimal threshold is obtained.

6. The rearview mirror retraction detection and warning method according to claim 1, characterized in that: The determination of the compatibility of the distance between the two rearview mirrors with the specifications of the parking spaces is specifically as follows: the number of pixels occupied by the horizontal track and the vehicle rearview mirror in the top view of the pre-parked vehicle is calculated, and the distance between the two rearview mirrors of the vehicle when the rearview mirrors are not retracted is calculated in proportion based on the actual distance between the horizontal tracks, and the width suitable for parking in the intelligent stereo garage is compared.

7. The rearview mirror retraction detection and warning method according to claim 1, characterized in that: At the entrance of the intelligent three-dimensional garage, the pre-parking vehicle completes the positioning operation, specifically: when the vehicle enters the garage, the front of the vehicle reaches the position of the infrared device 1, and the indicator light turns from red to green under the induction of the infrared device, indicating that the positioning is completed.

8. The rearview mirror retraction detection and warning method according to claim 1, characterized in that: The FPN-improved Faster RCNN model uses FPN to replace the original feature extraction part of the Faster RCNN model.

9. The rearview mirror retraction detection and warning method according to claim 1, characterized in that: The RPN sets a 3×3 convolution and two 1×1 convolutions, the former is used to concentrate features, and the latter two are used to generate the initially screened regions of interest.

10. A system for implementing the rearview mirror retraction detection and early warning method according to any one of claims 1 to 9, characterized in that: include: The pre-warehouse vehicle positioning system uses infrared devices to locate the pre-warehouse vehicle to ensure that the vehicle position conditions meet the requirements of rearview mirror detection; The rearview mirror detection system for pre-parked vehicles uses a camera to capture a bird's-eye view of the pre-parked vehicle and inputs the image into a pre-trained FPN-modified Faster RCNN model to detect and locate the position of the rearview mirror. The rearview mirror status judgment and warning system judges the status of the rearview mirror according to the output results of the rearview mirror detection system of the pre-parked vehicle, and determines whether to issue an alarm based on the comprehensive judgment results to remind the vehicle owner.

Images