Intelligent mobile vehicle inspection device and method based on Beidou positioning and visual SLAM

Abstract

The present application relates to the field of new urban management. In order to improve the efficiency of urban operation and management, a smart mobile inspection device and method based on Beidou positioning and visual SLAM are disclosed. The hardware components of the device include: Beidou high-precision combined navigation and positioning unit, which is used to obtain real-time high-precision position information of the mobile carrier unit according to the built-in RTK+IMU combined navigation and positioning module; real-time high-definition video acquisition unit, which is used to obtain environmental information within a preset range of the mobile carrier unit; computing power unit, which is used to monitor the environmental changes within a preset range in real time based on the real-time position information of the mobile carrier unit and the environmental information within the preset range, combined with the visual SLAM algorithm. The data communication unit is used to transmit data to the background service; the mobile carrier unit is used to carry the smart mobile inspection device. The high-precision positioning algorithm + real-time early warning system of this device has the effect of improving the overall management efficiency of the city and realizing the refinement and dynamic management of urban governance.

Description

Technical Field

[0001] The present invention relates to the field of smart city operation and management, and in particular to a smart mobile vehicle inspection device and method based on Beidou positioning and visual SLAM. Background Art

[0002] In the process of building a smart city, patrol personnel are responsible for daily inspections, reporting, verification, examination, and census of urban management incidents. They are the main source of problems discovered during inspections, and are highly dependent on the professional level of patrol personnel. They target some core issues in urban management, such as occupying the road for business, vehicles occupying the road, illegal parking, etc., to achieve intelligent supervision, information sharing, and business collaboration, so as to promptly discover, handle, and resolve various problems arising in urban operations.

[0003] At present, illegal parking violations are monitored 24 / 7 through the use of illegal parking balls, which capture graphic information of vehicle violations and handle the violations afterwards. There are also video poles that use deep learning algorithms, video stream analysis technology and AI recognition technology, and combine with the cloud to achieve unmanned and contactless parking fee collection, and interconnect with smart government platforms such as the public security, traffic police, and urban management to form urban big data. However, the management scope of illegal parking balls and video poles is limited. There are multiple parking spaces in the parking lot, and relevant data can only be temporarily obtained when the vehicle approaches a parking space.

[0004] Regarding the above-mentioned related technologies, the inventor believes that the related technologies still need to improve the efficiency of supervision when supervising urban violations. Summary of the invention

[0005] In order to improve the efficiency of supervision when supervising urban violations, the present application provides an intelligent mobile vehicle inspection device and method based on Beidou positioning and visual SLAM.

[0006] In the first aspect, the present application provides a smart mobile vehicle inspection device based on Beidou positioning and visual SLAM, which adopts the following technical solutions:

[0007] A smart mobile vehicle inspection device based on Beidou positioning and visual SLAM, comprising a computing unit, and a Beidou high-precision integrated navigation and positioning unit, a data communication unit, a real-time high-definition video acquisition unit and a mobile carrier unit connected to the computing unit;

[0008] The Beidou high-precision integrated navigation and positioning unit is used to obtain the real-time position information of the mobile carrier unit according to the preset RTK+IMU integrated navigation and positioning module;

[0009] The real-time high-definition video acquisition unit is used to obtain environmental information within a preset range of the mobile carrier unit;

[0010] The data communication unit is used to upload the real-time location information of the mobile carrier unit and the environmental information within a preset range of the mobile carrier unit to a preset back-end vehicle management service platform;

[0011] The computing unit is used to locate the mobile carrier unit in real time based on the real-time position information of the mobile carrier unit and the environmental information within a preset range of the mobile carrier unit;

[0012] The mobile carrier unit is respectively connected to the Beidou high-precision combined navigation and positioning unit, the computing power unit and the real-time high-definition video acquisition unit, and is used to carry the intelligent mobile vehicle inspection device for real-time movement.

[0013] By adopting the above technical solution and setting up a mobile carrier unit, the device has diversified forms and can be installed on mobile vehicles, such as electric bicycles, motorcycles, electric patrol cars, security vehicles and police cars, etc., to facilitate the normal vehicle inspection work of patrol personnel; the built-in RTK+IMU combined navigation and positioning module can achieve the centimeter-level positioning accuracy of the device; the front-end is configured with a multi-eye video acquisition unit to achieve visual SLAM capability, with high-precision positioning and street scene modeling capabilities, travel SLAM capability, ranging capability, mapping and memory capabilities; the Beidou high-precision combined navigation and positioning unit and the visual SLAM unit are integrated: since the accuracy of the visual SLAM unit is not high, the accuracy of the visual SLAM mapping is improved by integrating the data of the Beidou high-precision combined navigation and positioning unit; therefore, the supervision efficiency can be improved during the vehicle inspection process by patrol personnel.

[0014] Optionally, the device further includes a vehicle prediction module, a license plate detection and recognition module, and a visual SLAM module;

[0015] The visual SLAM module is communicatively connected to the real-time high-definition video acquisition unit, and is used to obtain SLAM positioning data information of the mobile carrier unit, and based on the environmental information within a preset range of the mobile carrier unit, perform SLAM mapping and locate the mobile carrier unit;

[0016] The vehicle prediction module is connected to the visual SLAM module for determining whether the distance difference of the mobile carrier unit is within a preset distance threshold range;

[0017] It is also connected to the real-time high-definition video acquisition unit for determining whether there is a vehicle in the real-time captured image;

[0018] The license plate detection and recognition module is communicatively connected with the vehicle pre-judgment module, and is used to detect and recognize the license plate of the mobile carrier unit in response to a license plate detection and recognition instruction, and report the license plate information.

[0019] By adopting the above technical solution, when the mobile carrier unit enters the parking space to park, the real-time position information of the mobile carrier unit and the distance between the mobile carrier unit and the parking space are obtained. When the distance between the mobile carrier unit and the parking space is less than a certain threshold, it means that the mobile carrier unit has entered the capture range, and the mobile carrier unit needs to be captured, and then it is determined whether the captured image contains a license plate. When a license plate is contained, the license plate information is analyzed and identified, and then the license plate information is reported. The mobile carrier unit provides effective and accurate management and evidence data for patrol personnel by capturing the license plate in real time and accurately binding the license plate and parking space information.

[0020] Optionally, the device further comprises a pre-processing module;

[0021] The pre-processing module is connected to the computing unit for obtaining the longitude and latitude information of each parking space;

[0022] It is also used to obtain each parking space number and map each of the parking space numbers to a preset system internal numbering system;

[0023] And used to burn a preset mobile carrier unit modification program into the computing power unit.

[0024] By adopting the above technical solution and setting up a preliminary preprocessing module, the main purpose of obtaining the latitude and longitude information of the parking space, the correspondence between the parking space number and the internal numbering system of the system, and the burning of the mobile carrier unit modification program is achieved. The preliminary preparation work is conducive to the smooth and efficient implementation of mobile vehicle inspection.

[0025] Optionally, the license plate detection and recognition module includes a license plate detection unit, a license plate correction unit, a text recognition unit, a license plate category determination unit and a result output unit;

[0026] The license plate detection unit is used to identify the license plate image information based on the real-time captured image information of the mobile carrier unit;

[0027] The license plate correction unit is used to generate corrected license plate image information based on the license plate image information;

[0028] The text recognition unit is used to generate license plate text result information based on the corrected license plate image information;

[0029] The license plate type identification unit is used to identify the color of the license plate based on the license plate image information and obtain the mobile carrier unit type information;

[0030] The result output unit is used to output the license plate text result information and the mobile carrier unit type information.

[0031] By adopting the above technical solution, after capturing the mobile carrier unit, the license plate image is extracted from the captured mobile carrier unit image, and then the license plate image is corrected to facilitate clear identification of the text image in the license plate. Then, by identifying the color of the license plate, for example, a blue license plate is an ordinary license plate, a green license plate is a new energy license plate, a white license plate is a military or civilian license plate, etc., the type of the mobile carrier unit can be determined. Finally, the mobile carrier unit type and the license plate text information are reported to the background, which can record the mobile carrier unit parked in the parking space and facilitate supervision by patrol personnel.

[0032] In the second aspect, the present application provides a smart mobile vehicle inspection method based on Beidou positioning and visual SLAM, which adopts the following technical solutions:

[0033] A smart mobile vehicle inspection method based on Beidou positioning and visual SLAM is applied to the above-mentioned smart mobile vehicle inspection device based on Beidou positioning and visual SLAM, comprising:

[0034] Carry out preliminary pre-processing of the intelligent mobile inspection device;

[0035] Acquire high-precision location information of mobile carrier units in real time through Beidou high-precision combined positioning unit;

[0036] Based on the high-precision position information of the mobile carrier unit, the environmental information within the preset range of the mobile carrier unit is obtained;

[0037] Obtain high-definition video data through a real-time high-definition video acquisition unit;

[0038] Acquire SLAM positioning data information of the mobile carrier unit, and perform SLAM mapping and positioning of the mobile carrier unit based on environmental information within a preset range of the mobile carrier unit;

[0039] The high-definition video data, SLAM mapping and data information for positioning the mobile carrier unit are transmitted to the preset back-end vehicle management service platform through the data transmission unit.

[0040] By adopting the above technical solution, the built-in RTK+IMU combined navigation and positioning module can achieve centimeter-level positioning accuracy of this device; by configuring a multi-eye video acquisition unit at the front end, the visual SLAM capability is realized, and it has the capabilities of high-precision positioning and street scene modeling, travel SLAM capability, ranging capability, mapping and memory capability; the Beidou high-precision combined navigation and positioning unit and the visual SLAM unit are integrated: since the accuracy of the visual SLAM unit is not high, the accuracy of the visual SLAM mapping is improved by integrating the data of the Beidou high-precision combined navigation and positioning unit; therefore, the supervision efficiency can be improved during the inspection of vehicles by patrol personnel.

[0041] Optionally, the step of performing preliminary preprocessing on the intelligent mobile vehicle inspection device includes:

[0042] Get the latitude and longitude information of each parking space;

[0043] Get each parking space number;

[0044] Mapping each of the parking space numbers to a preset system internal numbering system;

[0045] Burn the preset mobile carrier unit modification program into the preset computing power unit.

[0046] By adopting the above technical solution and setting up a preliminary preprocessing module, the main purpose of obtaining the latitude and longitude information of the parking space, the correspondence between the parking space number and the internal numbering system of the system, and the burning of the mobile carrier unit modification program is achieved. The preliminary preparation work is conducive to the smooth and efficient implementation of mobile vehicle inspection.

[0047] Optionally, the steps of performing SLAM mapping and positioning the mobile carrier unit include:

[0048] Obtain high-definition video data based on real-time high-definition video acquisition unit and process it through the front-end visual odometer;

[0049] Based on the result of the front-end visual odometer processing, performing back-end optimization processing;

[0050] Based on the result of the back-end optimization processing, SLAM mapping is performed;

[0051] Based on SLAM composition, loop detection is performed;

[0052] During positioning, the vehicle is positioned through the map after loop closure detection.

[0053] By adopting the above technical solution and configuring a multi-eye video acquisition unit at the front end, visual SLAM capability can be achieved, and the system has the capabilities of high-precision positioning and street scene modeling, as well as travel SLAM, ranging, mapping and memory capabilities. After SLAM mapping, loop detection is performed to improve the accuracy of mapping and the accuracy of subsequent vehicle positioning.

[0054] Optionally, the steps of performing SLAM mapping and positioning the mobile carrier unit include:

[0055] Obtaining the real-time location information of the mobile carrier unit and the distance difference information between the mobile carrier unit and the parking space;

[0056] Determining whether the distance difference is within a preset distance threshold range;

[0057] When the distance difference is within a preset distance threshold range, controlling a preset real-time high-definition video acquisition unit to obtain real-time captured image information of the mobile carrier unit;

[0058] Determining whether there is a vehicle in the real-time captured image;

[0059] When there is a vehicle in the real-time captured image, the preset license plate detection and recognition module is controlled to detect the license plate of the mobile carrier unit to identify the license plate information, and report the license plate information.

[0060] By adopting the above technical solution, when the mobile carrier unit enters the parking space to park, the real-time position information of the mobile carrier unit and the distance between the mobile carrier unit and the parking space are obtained. When the distance between the mobile carrier unit and the parking space is less than a certain threshold, it means that the mobile carrier unit has entered the capture range, and the mobile carrier unit needs to be captured, and then it is determined whether the captured image contains a license plate. When the license plate is contained, the license plate information is analyzed and identified, and then the license plate information is reported. The mobile carrier unit provides car owners with a quick way to find parking spaces by capturing the license plate in real time and accurately binding the license plate and parking space information, and provides patrol personnel with effective and accurate management and evidence data.

[0061] Optionally, the step of controlling a preset license plate detection and recognition module to detect and recognize the license plate of the mobile carrier unit includes:

[0062] Recognize the license plate image information based on the real-time captured image information of the mobile carrier unit;

[0063] generating corrected license plate image information based on the license plate image information, and identifying the color of the license plate based on the license plate image information, thereby acquiring mobile carrier unit type information;

[0064] Generate license plate text result information based on the corrected license plate image information;

[0065] Output the license plate text result information and the mobile carrier unit type information.

[0066] By adopting the above technical solution, after capturing the mobile carrier unit, the license plate image is extracted from the captured mobile carrier unit image, and then the license plate image is corrected to facilitate clear identification of the text image in the license plate. Then, by identifying the color of the license plate, for example, a blue license plate is an ordinary license plate, a green license plate is a new energy license plate, a white license plate is a military or civilian license plate, etc., the type of the mobile carrier unit can be determined. Finally, the mobile carrier unit type and the license plate text information are reported to the background, which can record the mobile carrier unit parked in the parking space and facilitate background supervision.

[0067] In summary, this application includes the following beneficial technical effects:

[0068] 1. By setting up a mobile carrier unit, the form is diversified and can be installed on mobile vehicles, such as electric bicycles, motorcycles, electric patrol cars, security vehicles and police cars, etc., which is convenient for the normal vehicle inspection work of patrol personnel; the built-in RTK+IMU combined navigation and positioning module can achieve the centimeter-level positioning accuracy of the device; through the front-end configuration of a multi-eye video acquisition unit, the visual SLAM capability is realized, and it has the ability of high-precision positioning and street scene modeling, travel SLAM capability, ranging capability, mapping and memory capability; integration of Beidou high-precision combined navigation and positioning unit and visual SLAM unit: due to the low accuracy of the visual SLAM unit, the accuracy of visual SLAM mapping is improved by integrating the data of Beidou high-precision combined navigation and positioning unit; therefore, during the inspection process of patrol personnel, the supervision efficiency can be improved.

[0069] 2. The mobile carrier unit provides car owners with a quick way to find parking spaces by capturing license plates in real time and accurately binding license plates to parking space information, and provides patrol personnel with effective and accurate management and evidence data. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 It is a schematic diagram of the hardware architecture of an intelligent mobile vehicle inspection device based on Beidou positioning and visual SLAM in an embodiment of the present application.

[0071] Figure 2 It is a flow chart of an intelligent mobile vehicle inspection method based on Beidou positioning and visual SLAM in an embodiment of the present application.

[0072] Figure 3 yes Figure 2 FIG. 5 is an expanded flowchart of step S100 in FIG.

[0073] Figure 4 yes Figure 2 The expansion process of step S500 Figure 1 .

[0074] Figure 5 yes Figure 2 The expansion process of step S500 Figure 2 .

[0075] Figure 6 yes Figure 5 FIG. 5 is an expanded flowchart of step S5C0 in FIG.

[0076] Explanation of the accompanying drawings: 1. Computing unit; 2. Beidou high-precision integrated navigation and positioning unit; 3. Real-time high-definition video acquisition unit; 4. Data communication unit; 5. Pre-processing module; 6. License plate detection and recognition module; 61. License plate detection unit; 62. License plate correction unit; 63. Text recognition unit; 64. License plate category determination unit; 65. Result output unit; 7. Vehicle prediction module; 8. Visual SLAM module; 9. Mobile carrier unit. DETAILED DESCRIPTION

[0077] The following is combined with Figure 1-6 This application is described in further detail.

[0078] Reference Figure 1 The embodiment of the present application discloses an intelligent mobile vehicle inspection device based on Beidou positioning and visual SLAM, including a computing unit 1, a Beidou high-precision combined navigation and positioning unit 2, a visual SLAM module 8, a mobile carrier unit 9, a data communication unit 4, a pre-processing module 5, a precision positioning module 2, a vehicle prediction module 7, a camera module 4, a control module 5 and a license plate detection and recognition module 6.

[0079] Beidou high-precision integrated navigation and positioning unit 2 is used to obtain the real-time position information of the mobile carrier unit according to the preset RTK+IMU integrated navigation and positioning module. The mobile carrier unit in this solution can be a bicycle, a car or other types of vehicles.

[0080] RTK (Real-time kinematic) carrier phase differential technology is a differential method for real-time processing of carrier phase observations of two measurement stations. The carrier phase collected by the base station is sent to the user receiver to calculate the coordinates. This is a new and commonly used satellite positioning measurement method. In the past, static, fast static, and dynamic measurements all required post-calculation to obtain centimeter-level accuracy, while RTK is a measurement method that can obtain centimeter-level positioning accuracy in real time in the field.

[0081] The Inertial Navigation System (INSERTIAL NAVIGATION SYSTEM) is an autonomous navigation system that does not rely on external information and does not radiate energy to the outside (like radio navigation). It mainly uses an inertial measurement unit (IMU). Its working environment includes not only the air and the ground, but also underwater. The basic working principle of inertial navigation is based on Newton's laws of mechanics. By measuring the acceleration of the carrier in the inertial reference system, integrating it over time, and transforming it into the navigation coordinate system, it is possible to obtain information such as speed, yaw angle, and position in the navigation coordinate system. Its advantages are that it does not require time to work, the installation position is arbitrary, and the positioning range is full, but the positioning accuracy is not high, and the error diverges over time. It is complementary to the GNSS navigation system.

[0082] Single RTK positioning cannot provide continuous positioning and navigation data in the case of obstruction. This module integrates GNSS receivers and IMUs that support BDS, GPS, GALILEO, GLONASS, and QZSS, and adopts multi-band RTK and combined navigation technology to achieve centimeter-level accuracy. It can simultaneously receive and resolve the latest multi-frequency global navigation and positioning satellite signals, and can still provide highly reliable positioning data in the case of severe satellite signal obstruction, providing uninterrupted navigation and positioning output.

[0083] This device adopts a high-precision, high-reliability centimeter-level combined navigation and positioning module, and is highly integrated, small in size, good in I / O compatibility, and low in power consumption. It is mainly used to locate the real-time position of the mobile carrier unit and compare it with the real-time position of the parking space. This module integrates GNSS receivers and IMUs that support BDS, GPS, GALILEO, GLONASS, and QZSS, and uses multi-band RTK and combined navigation technology to achieve centimeter-level accuracy. It can simultaneously receive and resolve the latest multi-frequency global navigation and positioning satellite signals, and can still provide highly reliable positioning data in the case of severe satellite signal obstruction, and provide uninterrupted navigation and positioning output to achieve centimeter-level, millimeter-level and nanometer-level high-precision positioning accuracy; the module performance parameters are as follows:

[0084] GNSS performance:

[0085] Tracking frequency: GPS / QZSS, BEIDOU, GALILEO, GLONASS;

[0086] Horizontal accuracy: ≤8mm+1ppm;

[0087] Start-up time: cold start ≤ 27s, hot start 2s;

[0088] Fixed solution convergence time ≤ 10s;

[0089] Path-2-path accuracy: sub-meter level;

[0090] Antenna: single antenna;

[0091] Baseline length: 40km fixed solution;

[0092] Capture sensitivity: -148dBm;

[0093] Tracking sensitivity: -165dBm;

[0094] Carrier phase observation accuracy: <0.01cycle;

[0095] Navigation output frequency: Up to 10Hz;

[0096] IMU Performance:

[0097] Gyroscope: mileage ±1000deg / s, bias stability ±4.5deg / h, angle random walk 0.17deg / √h;

[0098] Accelerometer: mileage ±16g, bias stability ±0.3mg, speed random walk 0.04m / s / √h;

[0099] Roll and pitch accuracy: ≤0.02° (1σ);

[0100] Heading accuracy: ≤0.2° (1σ);

[0101] Inertial estimation accuracy: ≤1%*D (1σ, vehicle-mounted);

[0102] Data sampling rate: 50Hz;

[0103] Map matching technology MM (Map matching) combines user location information and map data to calculate the user's exact location on the road on the map, assisting in the precise control of in-vehicle navigation.

[0104] The computing unit 1 can be an algorithm controller for running programs. It is connected to the Beidou high-precision integrated navigation and positioning unit 2 in communication, and is used to obtain environmental information within a preset range of the mobile carrier unit based on the real-time position information of the mobile carrier unit; realize the Beidou high-precision integrated navigation RTK+IMU positioning algorithm solution, realize environmental perception and video acquisition, and perform artificial intelligence algorithms and corresponding processing capabilities.

[0105] The visual SLAM module 8 is connected to the computing unit 1 for communication, and is used to obtain the SLAM positioning data information of the mobile carrier unit, and to perform SLAM mapping and positioning of the mobile carrier unit based on the environmental information within a preset range of the mobile carrier unit.

[0106] Visual SLAM module 8 uses the ORB-SLAM algorithm and is a complete SLAM system, including visual odometer, tracking, and loop monitoring. It is a monocular SLAM system based entirely on sparse feature points, and also has interfaces for monocular, binocular, and RGBD cameras. Its core is to use ORB (Orinted FAST and BRIEF) as the core feature of the entire visual SLAM; ORB-SLAM basically continues the algorithm framework of PTAM, but has made improvements to most of the components in the framework, which can be summarized into four main points:

[0107] 1. ORB-SLAM uses ORB features. The feature matching and relocation based on ORB descriptions have better perspective invariance than PTAM. In addition, the feature matching efficiency of newly added 3D points is higher, so the scene can be expanded more timely. Whether the scene is expanded in time determines whether the subsequent frames can be tracked stably.

[0108] 2. ORBSLAM adds a loop detection and closure mechanism to eliminate error accumulation. The system uses the same method as relocalization to detect loops (matching common points on keyframes on both sides of the loop) and closes the loop through pose graph optimization.

[0109] 3. PTAM requires the user to specify 2 frames to initialize the system. There must be enough common points and enough translation between the 2 frames. The translation motion provides parallax for these common points. Only with enough parallax can the accurate 3D position be triangulated. ORB-SLAM automatically selects the 2 frames for initialization by detecting the parallax.

[0110] 4. When PATM expands the scene, it also requires the newly added keyframes to provide sufficient parallax, which makes the scene difficult to expand. ORB-SLAM adopts a more robust keyframe and 3D point selection mechanism - first use loose judgment conditions to add new keyframes and 3D points as soon as possible to ensure robust tracking of subsequent frames; then use strict judgment conditions to delete redundant keyframes and unstable 3D points to ensure the efficiency and accuracy of BA.

[0111] ORB-SLAM consists of three major blocks and three processes running simultaneously: the first block is tracking, the second block is mapping, and the third block is closed-loop detection.

[0112] 1. Tracking

[0113] The main work of this part is to extract ORB features from the image, perform pose estimation based on the previous frame, or initialize the position through global relocalization, then track the reconstructed local map, optimize the position, and then determine the new keyframe according to some rules;

[0114] 2. Local Mapping

[0115] This part mainly completes the local map construction, including inserting keyframes, verifying and filtering the recently generated map points, generating new map points, using local bundle adjustment (Local BA), and finally filtering the inserted keyframes to remove redundant keyframes.

[0116] 3. Loop Closing

[0117] This part is mainly divided into two processes, namely closed loop detection and closed loop correction. Closed loop detection first uses WOB for detection, and then calculates similarity transformation with the Sim3 algorithm. Closed loop correction mainly involves closed loop fusion and Essential Graph optimization.

[0118] ORB-SLAM algorithm advantages:

[0119] A visual SLAM system with excellent code structure, very suitable for porting to actual projects.

[0120] Using g2o as a backend optimization tool can effectively reduce the estimation errors of feature point positions and the position of the image itself.

[0121] The use of DBOW reduces the amount of computation required to find features, while loop matching and relocalization have better effects. Relocalization: For example, when the device encounters some unexpected situations, its data flow is suddenly interrupted. Under the ORB-SLAM algorithm, the device can be relocated in the map in a short time.

[0122] A similar "survival of the fittest" approach is used to select key frames, improving the robustness of system tracking and the sustainable operation of the system;

[0123] Since the accuracy of the visual SLAM unit is not high, or it produces cumulative errors and drift, the accuracy of visual SLAM mapping can be improved by integrating the high-precision positioning data of the Beidou high-precision integrated navigation and positioning unit.

[0124] The mobile carrier unit 9 is respectively connected to the Beidou high-precision integrated navigation and positioning unit 2, the computing unit 1 and the visual SLAM module 8, and is used to carry the intelligent mobile vehicle inspection device for real-time movement.

[0125] The mobile carrier unit 9 can be: a mobile vehicle, such as an electric bicycle, a motorcycle, an electric patrol car, a security vehicle, a police car and other mobile carriers.

[0126] The preliminary preprocessing module 5 is connected to the control module 5 for obtaining the latitude and longitude information of each parking space; it is also used to obtain the number of each parking space and map each parking space number to a preset system internal numbering system; and it is used to burn the preset mobile carrier unit modification program into the control module 5.

[0127] In the early stage, it is necessary to manually measure the latitude and longitude corresponding to each parking space, and correspond the parking space number with its own internal number. When the smart mobile inspection device is inspecting, the vehicle-mounted GPS is compared with the GPS information of each parking space measured in advance. When the vehicle drives to the corresponding position, the camera takes a snapshot (due to GPSRTK+IMU positioning, the smart mobile inspection device can trigger the snapshot when it reaches the head and tail of the parking space, and the first camera of the three cameras is used to capture the snapshot respectively, and there is a fill light at night). When there is a vehicle in the captured photo, the vehicle will be detected, and the license plate will be detected for license plate recognition. When there is a car parked in a normal parking space, the vehicle picture, license plate information, parking space number, time, vehicle type and other information will be reported after the recognition result. If there is an abnormal situation such as the license plate being blocked, the vehicle picture, parking space number, time, etc. will be returned to notify the on-site personnel to go to the corresponding point for inspection.

[0128] Beidou high-precision integrated navigation and positioning unit 2 is also used to obtain the real-time position information of the mobile carrier unit and the distance difference information between the mobile carrier unit and the parking space. The high-precision positioning device GNSS module is used to locate the real-time position of the mobile carrier unit and compare it with the real-time position of the parking space, achieving RTK+IMU, millimeter-level and nanometer-level high-precision positioning accuracy.

[0129] The accuracy of ordinary positioning services is usually at the meter level, which cannot meet more sophisticated scenarios. High-precision positioning uses RTK technology. The base station sends the observed carrier phase measurement value and its own coordinate information to the mobile station in real time through the data link. The mobile station receives the carrier phase of the GPS satellite signal and the carrier phase measurement value from the base station. By using the spatial correlation of the observation error between the base station and the mobile station, most of the errors in the mobile station's observation data are reduced by differential means, thereby achieving RTK+IMU positioning.

[0130] The vehicle prediction module 7 is connected to the Beidou high-precision integrated navigation and positioning unit 2 for communication, and is used to determine whether the distance difference is within a preset distance threshold range.

[0131] The control module 5 is connected to the vehicle prediction module 7 for sending a camera instruction to the camera module 4 when the distance difference is within a preset distance threshold range; the control module 5 can be a hardware structure or software structure such as an industrial computer or a CPU for running a computer program.

[0132] The camera module 4 is connected to the control module 5 in communication, and is used to respond to the camera instruction to obtain the real-time captured image information of the mobile carrier unit; preferably, it is a license plate capture camera, which is composed of three license plate capture cameras, which are respectively aimed at the front, body and rear of the vehicle, and record and capture vehicle data in real time. The module performance parameters are as follows:

[0133] Photosensitive size: 1 / 2.6''

[0134] Focal length: 3.6mm

[0135] Working temperature: -20 degrees to 70 degrees

[0136] Resolution: Maximum support 1920*1080

[0137] Maximum frame rate: 100fps (MJPG)

[0138] The vehicle prediction module 7 is also connected to the camera module 4 for communication, and is also used to determine whether there is a vehicle in the real-time captured image.

[0139] The control module 5 is also in communication connection with the vehicle prediction module 7, and is used to send a license plate detection and recognition instruction to the license plate detection and recognition module 6 when a vehicle exists in the real-time captured image.

[0140] The data communication unit 4 is used to upload the real-time position information of the mobile carrier unit, the distance difference information between the mobile carrier unit and the parking space, and the real-time captured image information of the mobile carrier unit to a preset back-end vehicle management service platform.

[0141] The license plate detection and recognition module 6 is in communication connection with the control module 5, and is used to detect and recognize the license plate of the mobile carrier unit in response to the license plate detection and recognition instruction, and report the license plate information.

[0142] The license plate detection and recognition module 6 includes a license plate detection unit 61, a license plate correction unit 62, a text recognition unit 63, a license plate category discrimination unit 64 and a result output unit 65. The license plate recognition algorithm is aimed at the pictures captured in the outdoor mobile state, which often have backlighting and blurring. In actual use, the camera lens and image preprocessing technologies are improved to deal with them. The designed license plate detection, correction and text recognition models also use end-to-end network training to improve the robustness of the model.

[0143] The license plate detection unit 61 is used to identify the license plate image information based on the real-time captured image information of the mobile carrier unit.

[0144] The license plate correction unit 62 is used to generate corrected license plate image information based on the license plate image information.

[0145] The text recognition unit 63 is used to generate license plate text result information based on the corrected license plate image information.

[0146] The license plate type discrimination unit 64 is used to identify the color of the license plate based on the license plate image information and obtain the mobile carrier unit type information. For example, after the mobile carrier unit is captured, the license plate image is extracted from the captured mobile carrier unit image, and then the license plate image is corrected to facilitate clear identification of the text image in the license plate, and then the color of the license plate is identified at the same time, such as a blue license plate for an ordinary license plate, a green license plate for a new energy license plate, a white license plate for a military or civilian license plate, etc., so as to achieve the effect of determining the type of the mobile carrier unit, and finally report the mobile carrier unit type and the license plate text information to the background, which can record the mobile carrier unit parked in the parking space and facilitate background supervision.

[0147] The result output unit 65 is used to output the license plate text result information and the mobile carrier unit type information.

[0148] The inspection device is mainly in the form of a video intelligent mobile inspection device. The intelligent mobile inspection device only uploads pictures (saving traffic), and the video is saved locally. If necessary, it can be copied through a network connection. The intelligent mobile inspection device can work normally for 8 hours, depending on the mileage. The terminal equipment is highly integrated and uniformly installed on the mobile vehicle, which can be easily installed and debugged on site and is simple to maintain; with a car in hand, the moving driving recorder greatly improves the parking management area, adopts GPS+Beidou dual-star differential positioning, with an accuracy of RTK+IMU, and an accuracy rate of more than 99% in identifying parking spaces; uses high-definition sports camera license plate recognition technology to obtain vehicle information, equipped with fill lights, and can ensure a license plate recognition rate of more than 98% in various lighting environments; parking records are automatically identified and uploaded to the system through 4G / Internet of Things, without human interference, and the automatically generated charging evidence chain is complete, isolating disputes.

[0149] The implementation principle of an intelligent mobile vehicle inspection device based on Beidou positioning and visual SLAM in the embodiment of the present application is as follows:

[0150] The high-precision position information of the mobile carrier unit is obtained in real time through the Beidou high-precision combined positioning unit; based on the high-precision position information of the mobile carrier unit, the environmental information within the preset range of the mobile carrier unit is obtained; the SLAM positioning data information of the mobile carrier unit is obtained, and based on the environmental information within the preset range of the mobile carrier unit, SLAM mapping and positioning of the mobile carrier unit are performed; when the mobile carrier unit enters the parking space and parks, the real-time position information of the mobile carrier unit and the distance between the mobile carrier unit and the parking space are obtained. When the distance between the mobile carrier unit and the parking space is less than a certain threshold, it means that the mobile carrier unit has entered the capture range, and the mobile carrier unit needs to be captured, and then it is determined whether the captured image contains a license plate. When a license plate is contained, the license plate information is analyzed and identified, and then the license plate information is reported. The mobile carrier unit provides effective and accurate management and evidence data for patrol personnel by capturing the license plate in real time and accurately binding the license plate and parking space information.

[0151] Based on the above hardware architecture, refer to Figure 2 The embodiment of the present application also discloses a smart mobile vehicle inspection method based on Beidou positioning and visual SLAM, including steps S100~S600:

[0152] Step S100: Pre-processing the intelligent mobile vehicle inspection device.

[0153] Step S200: Acquire high-precision position information of the mobile carrier unit in real time through the Beidou high-precision combined positioning unit.

[0154] Step S300: Based on the high-precision position information of the mobile carrier unit, obtain environmental information within a preset range of the mobile carrier unit.

[0155] Step S400: acquiring high-definition video data through a real-time high-definition video acquisition unit.

[0156] Step S500: Acquire SLAM positioning data information of the mobile carrier unit, and perform SLAM mapping and positioning of the mobile carrier unit based on environmental information within a preset range of the mobile carrier unit.

[0157] Step S600: The high-definition video data, the data information obtained through SLAM mapping and the positioning of the mobile carrier unit are transmitted to a preset back-end vehicle management service platform through a data transmission unit.

[0158] Reference Figure 3 The steps for pre-processing the intelligent mobile inspection device include:

[0159] Step S110: Obtain the longitude and latitude information of each parking space.

[0160] Step S120: Obtain each parking space number.

[0161] Step S130: Map each of the parking space numbers to a preset system internal numbering system.

[0162] Specifically, for example, the parking space number is 01, and the corresponding number in the system's internal numbering system is 0001. By establishing a corresponding relationship, the offline parking space number can be synchronized with the online system.

[0163] Step S140: Burn the preset mobile carrier unit modification program into the preset computing unit.

[0164] Reference Figure 4 The steps of performing SLAM mapping and positioning the mobile carrier unit include steps S510 to S550:

[0165] Step S510: Obtain high-definition video data based on the real-time high-definition video acquisition unit and process it through the front-end visual odometer.

[0166] Step S520: performing back-end optimization processing based on the result of the front-end visual odometer processing.

[0167] Step S530: Based on the result of the back-end optimization processing, SLAM mapping is performed.

[0168] Step S540: Perform loop detection based on SLAM mapping.

[0169] Step S550: During positioning, the vehicle is positioned through the map after loop closure detection.

[0170] Reference Figure 5 The steps of SLAM mapping and positioning the mobile carrier unit include steps S5A0 to S5E0:

[0171] Step S5A0: Acquire the real-time position information of the mobile carrier unit and the distance difference information between the mobile carrier unit and the parking space.

[0172] Step S5B0: Determine whether the distance difference is within a preset distance threshold range.

[0173] Step S5C0: When the distance difference is within the preset distance threshold range, the preset real-time high-definition video acquisition unit 4 is controlled to obtain the real-time captured image information of the mobile carrier unit. For example, the distance threshold range between the mobile carrier unit and the parking space is 30-50 cm. When it is within this range, the function of capturing the image is activated.

[0174] When the distance difference is within the preset distance threshold range, no processing is performed.

[0175] Step S5D0: Determine whether there is a vehicle in the real-time captured image.

[0176] Step S5E0: When there is a vehicle in the real-time captured image, the preset license plate detection and recognition module 6 is controlled to detect the license plate of the mobile carrier unit to identify the license plate information, and report the license plate information.

[0177] When the license plate detection and recognition module 6 fails to recognize the license plate information of the mobile carrier unit, it reports and notifies the on-site personnel.

[0178] Reference Figure 6 The step S5C0 of controlling the preset real-time high-definition video acquisition unit 4 to obtain the real-time captured image information of the mobile carrier unit includes steps S5C1 to S5C4:

[0179] Step S5C1: recognizing the license plate image information based on the real-time captured image information of the mobile carrier unit.

[0180] Step S5C2: Generate corrected license plate image information based on the license plate image information, and identify the color of the license plate based on the license plate image information, thereby obtaining the mobile carrier unit type information.

[0181] Step S5C3: Generate license plate text result information based on the corrected license plate image information.

[0182] Step S5C4: Output the license plate text result information and the mobile carrier unit type information.

[0183] The steps of identifying the license plate image information based on the real-time captured image information of the mobile carrier unit include:

[0184] Based on the real-time captured image of the mobile carrier unit, the position information of the license plate in the real-time captured image is obtained.

[0185] Based on the position information of the license plate in the real-time captured image, the confidence information of each license plate character is obtained.

[0186] Based on the confidence information of each license plate character, the category of each license plate character is determined.

[0187] Based on the position information of the license plate and the confidence information of each license plate character, the license plate image information is obtained.

[0188] Specifically, a real-time captured image is input, and the license plate detection is first performed on it to obtain the approximate position of the license plate in the image, using the Yolo series of neural network algorithms.

[0189] The Yolo algorithm, whose full name is You Only Look Once: Unified, Real-Time Object Detection, You Only Look Once means that only one CNN operation is required, Unified means that this is a unified framework that provides end-to-end prediction, and Real-Time reflects the fast speed of the Yolo algorithm. The Yolo algorithm uses a separate CNN model to achieve end-to-end object detection. First, the input image is resized to 448x448, then sent to the CNN network, and finally the network prediction results are processed to obtain the detected target. Compared with the R-CNN algorithm, it is a unified framework, it is faster, and the training process of Yolo is also end-to-end.

[0190] Specifically, Yolo's CNN network divides the input image into an SxS grid, and then each cell is responsible for detecting the targets whose center points fall within the grid. Each cell predicts B bounding boxes and the confidence score of the bounding box. The so-called confidence actually includes two aspects: one is the possibility that the bounding box contains the target, and the other is the accuracy of the bounding box. The former is recorded as Pr (object). When the bounding box is the background (that is, it does not contain the target), Pr (object) = 0, and when the bounding box contains the target, Pr (object) = 1. The accuracy of the bounding box can be represented by the IOU (intersection over union) of the predicted box and the actual box (ground truth), recorded as IOU_pred^truth. Therefore, the confidence can be defined as Pr (object) * IOU_pred^truth. Many people may regard Yolo's confidence as the probability of whether the bounding box contains the target, but in fact it is the product of two factors, and the accuracy of the predicted box is also reflected in it. The size and position of the bounding box can be represented by 4 values: (x, y, w, h), where (x, y) is the center coordinate of the bounding box, and w and h are the width and height of the bounding box. Another point to note is that the predicted value of the center coordinate (x, y) is the offset value relative to the coordinate point of the upper left corner of each cell, and the unit is relative to the cell size. The predicted values ​​of w and h of the bounding box are the ratio of the width and height of the entire image, so theoretically the size of the 4 elements should be in the range of [0, 1]. In this way, the predicted value of each bounding box actually contains 5 elements: (x, y, w, h, c), of which the first 4 represent the size and position of the bounding box, and the last value is the confidence.

[0191] There is also the classification problem. For each cell, it is necessary to predict C category probability values, which represent the probability that the target of the bounding box predicted by the cell belongs to each category. However, these probability values ​​are actually conditional probabilities under the confidence of each bounding box, that is, Pr(class_i│object). It is worth noting that no matter how many bounding boxes a cell predicts, it only predicts one set of category probability values.

[0192] The bounding box category confidence represents the probability that the object in the bounding box belongs to each category and how well the bounding box matches the object.

[0193] To summarize, each cell needs to predict (B*5+C) values. If the input image is divided into an SxS grid, the final predicted value is a tensor of size SxSx(B*5+C).

[0194] Network Design:

[0195] Yolo uses a convolutional network to extract features, and then uses a fully connected layer to get the predicted value. The network structure refers to the GooLeNet model, which contains 24 convolutional layers and 2 fully connected layers. For the convolutional layer, 1x1 convolution is mainly used for channel reduction, followed by 3x3 convolution. For the convolutional layer and the fully connected layer, the Leaky ReLU activation function is used: max(x, 0.1x). However, the last layer uses a linear activation function.

[0196] You can see that the final output of the network is a 7x7x30 size tensor. For each cell, the first 20 elements are the category probability values, then 2 elements are the bounding box confidence, the two are multiplied to get the category confidence, and the last 8 elements are the (x, y, w, h) of the bounding box.

[0197] Network training:

[0198] Before training, it was pre-trained on ImageNet, and its pre-trained classification model was adopted Figure 1 The first 20 convolutional layers are added, and then an average-pool layer and a fully connected layer are added. After pre-training, 4 randomly initialized convolutional layers and 2 fully connected layers are added on top of the 20 convolutional layers obtained by pre-training. Since detection tasks generally require higher-resolution images, the network input is increased from 224x224 to 448x448.

[0199] Network Prediction:

[0200] According to the previous analysis, the final network output is 7x7x30, but we can divide it into three parts: the category probability part is [7, 7, 20], the confidence part is [7, 7, 2], and the bounding box part is [7, 7, 2, 4] (for this part, the true value is calculated based on the original image). Then multiply the first two items (the matrix [7, 7, 20] multiplied by [7, 7, 2] can fill one dimension each to complete [7, 7, 1, 20] x [7, 7, 2, 1]) to get the category confidence value [7, 7, 2, 20]. Here, a total of 7*7*2=98 bounding boxes are predicted.

[0201] The Yolo algorithm first uses NMS and then determines the category of each box. For 98 boxes, the values ​​less than the confidence threshold are first set to 0, and then NMS is used on the confidence values ​​by category. Here, the NMS processing result is not to eliminate, but to set its confidence value to 0. Finally, the category of each box is determined, and the detection result is output when its confidence value is not 0.

[0202] The step of generating corrected license plate image information based on the license plate image information includes:

[0203] Based on the license plate image information, the scaled license plate image information is obtained.

[0204] After processing the scaled license plate image information using the preset WPOD-NET network, a central virtual rectangular frame with a fixed size is obtained.

[0205] Determine whether the probability that the central virtual rectangular frame contains the license plate characters is higher than a preset detection threshold.

[0206] If so, an affine matrix for transforming the virtual rectangle into the license plate area is constructed using preset partial regression parameters, thereby obtaining a license plate image that is corrected in both the horizontal and vertical directions.

[0207] License plate correction also adopts an end-to-end correction method, using the WPOD-NET network. The input is the result of license plate detection, and the output is the corrected license plate image.

[0208] Initially, the output of the vehicle detection module is scaled and fed into WPOD. The forward pass yields an 8-channel feature map containing the object / non-object probabilities and affine transformation parameters. To extract the distorted specific vehicle (LP), an imaginary rectangular box of fixed size (m, n) around the center of the cell is first considered. If the probability that the box contains an object is higher than a given detection threshold, some of the regression parameters are used to construct an affine matrix that transforms the imaginary square into the LP region. Therefore, we can easily correct the LP into horizontally and vertically aligned objects.

[0209] Network structure:

[0210] The proposed architecture has a total of 21 convolutional layers, 14 of which are contained within residual blocks. The size of all convolutional filters is fixed to 3*3. ReLU activation function is used throughout the network except for the detection module. There are four max pooling layers of size 2*2 and stride 2, which reduce the input dimension by a factor of 16. Finally, the detection module has two parallel convolutional layers: (i) one for inferring probability values ​​activated by a softmax function; (ii) the other for regressing affine parameters without an activation function (or equivalently, using the identity function F(x) = x as the activation function).

[0211] Network training:

[0212] In order to train the proposed WPOD-NET, we created a dataset containing a large number of vehicle images and also used a dataset enhancement method. The specific enhancement methods are as follows:

[0213] Correction: Assuming that LP is located on a plane, correct the entire image based on the annotation of LP;

[0214] Aspect ratio: The LP aspect ratio is randomly set in the interval [2, 4] to adapt to the size of different regions;

[0215] Centering: LP center becomes the image center;

[0216] Scale: Scale the LP so that its width is between 40px and 280px (experimentally set this based on the readability of the LP);

[0217] Rotation: performs 3D rotations with randomly selected angles to account for various camera setups;

[0218] Mirror: Perform mirror operation with a probability of 50%;

[0219] Translation: Randomly translate the license plate from the center of the image, limiting its range after translation to a square of 208 * 208;

[0220] Cropping: Considering the LP center before translation, we crop a 208×208 region around it;

[0221] Color transformation: modify the color in HSV color space;

[0222] Annotation: The positions of the four LP corners are adjusted by applying the same spatial transformation used to augment the input image.

[0223] The steps of generating the license plate text result information include:

[0224] Using the preset deep convolutional layer structure, features are extracted from the rectified license plate image information to obtain a feature map, which contains a feature sequence.

[0225] The preset bidirectional cyclic difference structure is used to predict the feature sequence, output the distribution information of the predicted label, and memorize and store each feature vector in the feature sequence.

[0226] Using the preset transcription layer structure, the distribution information of the predicted labels is converted into the final label sequence, and the final label sequence constitutes the license plate text result information.

[0227] The license plate type is identified using a common convolutional neural network (CNN) algorithm. The input is the captured license plate image, and the output is the license plate color. The vehicle type is determined based on the license plate color.

[0228] CNN is a type of neural network. Its weight-sharing network structure makes it more similar to biological neural networks, reducing the complexity of the network model and the number of weights. This advantage is more obvious when the network input is a multi-dimensional image, so that the image can be directly used as the input of the network, avoiding the complex feature extraction and data reconstruction process in traditional recognition algorithms. It has many advantages in two-dimensional image processing, such as the network can automatically extract image features including color, texture, shape and image topology; in processing two-dimensional image problems, especially in identifying displacement, scaling and other forms of distortion invariance, it has good robustness and computational efficiency.

[0229] CNN has some advantages that traditional technologies do not have: good fault tolerance, parallel processing and self-learning capabilities, can handle problems with complex environmental information, unclear background knowledge, and unclear reasoning rules, allows samples to have large defects and distortions, runs fast, has good adaptive performance, and has high resolution. It integrates feature extraction functions into multi-layer perceptrons by restructuring and reducing weights, omitting the complex image feature extraction process before recognition.

[0230] The above are all preferred embodiments of the present application, and the protection scope of the present application is not limited thereto. Therefore, any equivalent changes made according to the structure, shape, and principle of the present application should be included in the protection scope of the present application.

Claims

1. An intelligent mobile vehicle inspection device based on Beidou positioning and visual SLAM, characterized by: It comprises a computing unit (1), and a Beidou high-precision integrated navigation and positioning unit (2), a data communication unit (4), a real-time high-definition video acquisition unit (3) and a mobile carrier unit (9) which are communicatively connected to the computing unit (1); The Beidou high-precision integrated navigation and positioning unit (2) is used to obtain real-time position information of the mobile carrier unit according to a preset RTK+IMU integrated navigation and positioning module; The real-time high-definition video acquisition unit (3) is used to obtain environmental information within a preset range of the mobile carrier unit; The data communication unit (4) is used to upload the real-time location information of the mobile carrier unit and the environmental information within a preset range of the mobile carrier unit to a preset back-end vehicle management service platform; The computing unit (1) is used to locate the mobile carrier unit in real time based on the real-time position information of the mobile carrier unit and the environmental information within a preset range of the mobile carrier unit; The mobile carrier unit (9) is respectively connected to the Beidou high-precision integrated navigation and positioning unit (2), the computing unit (1) and the real-time high-definition video acquisition unit (3), and is used to carry the intelligent mobile vehicle inspection device for real-time movement; the device also includes a vehicle pre-judgment module (7), a license plate detection and recognition module (6) and a visual SLAM module (8); The visual SLAM module (8) is communicatively connected to the real-time high-definition video acquisition unit (3) for acquiring SLAM positioning data information of the mobile carrier unit, and performing SLAM mapping and positioning of the mobile carrier unit based on environmental information within a preset range of the mobile carrier unit; The vehicle pre-judgment module (7) is communicatively connected to the visual SLAM module (8) and is used to determine whether the distance difference of the mobile carrier unit is within a preset distance threshold range; It is also connected to the real-time high-definition video acquisition unit (3) for determining whether a vehicle exists in the real-time captured image; The license plate detection and recognition module (6) is in communication connection with the vehicle pre-judgment module (7), and is used to detect and recognize the license plate of the mobile carrier unit in response to the license plate detection and recognition instruction, and report the license plate information; The device also includes a pre-processing module (5); The pre-processing module (5) is connected to the computing unit (1) for obtaining the longitude and latitude information of each parking space; It is also used to obtain each parking space number and map each of the parking space numbers to a preset system internal numbering system; and for burning a preset mobile carrier unit modification program into the computing power unit (1); The license plate detection and recognition module (6) comprises a license plate detection unit (61), a license plate correction unit (62), a text recognition unit (63), a license plate type determination unit (64) and a result output unit (65); The license plate detection unit (61) is used to identify the license plate image information based on the real-time captured image information of the mobile carrier unit; The license plate correction unit (62) is used to generate corrected license plate image information based on the license plate image information; The character recognition unit (63) is used to generate license plate character result information based on the corrected license plate image information; The license plate type identification unit (64) is used to identify the color of the license plate based on the license plate image information and obtain the mobile carrier unit type information; The result output unit (65) is used to output the license plate text result information and the mobile carrier unit type information.

2. An intelligent mobile vehicle inspection method based on Beidou positioning and visual SLAM, characterized by: The intelligent mobile vehicle inspection device based on Beidou positioning and visual SLAM as described in claim 1 above comprises: Carry out preliminary pre-processing of the intelligent mobile inspection device; Acquiring high-precision position information of a mobile carrier unit (9) in real time through a Beidou high-precision combined positioning unit; Based on the high-precision position information of the mobile carrier unit (9), obtaining environmental information within a preset range of the mobile carrier unit (9); Acquiring high-definition video data through a real-time high-definition video acquisition unit (3); Acquire SLAM positioning data information of the mobile carrier unit, and perform SLAM mapping and positioning of the mobile carrier unit based on environmental information within a preset range of the mobile carrier unit; The high-definition video data, SLAM mapping and data information for positioning the mobile carrier unit are transmitted to the preset back-end vehicle management service platform through the data transmission unit.

3. According to claim 2, a smart mobile vehicle inspection method based on Beidou positioning and visual SLAM is characterized in that: The steps of pre-processing the intelligent mobile vehicle inspection device include: Get the latitude and longitude information of each parking space; Get each parking space number; Mapping each of the parking space numbers to a preset system internal numbering system; Burn the preset mobile carrier unit modification program into the preset computing power unit (1).

4. According to claim 2, a smart mobile vehicle inspection method based on Beidou positioning and visual SLAM is characterized in that: The steps of performing SLAM mapping and positioning the mobile carrier unit include: Obtaining high-definition video data based on a real-time high-definition video acquisition unit (3) and processing it through a front-end visual odometer; Based on the result of the front-end visual odometer processing, performing back-end optimization processing; Based on the result of the back-end optimization processing, SLAM mapping is performed; Based on SLAM composition, loop detection is performed; During positioning, the vehicle is positioned through the map after loop closure detection.

5. The intelligent mobile vehicle inspection method based on Beidou positioning and visual SLAM according to claim 2 is characterized in that: The steps of performing SLAM mapping and positioning the mobile carrier unit include: Obtaining the real-time location information of the mobile carrier unit and the distance difference information between the mobile carrier unit and the parking space; Determining whether the distance difference is within a preset distance threshold range; When the distance difference is within a preset distance threshold range, controlling a preset real-time high-definition video acquisition unit (3) to acquire real-time captured image information of the mobile carrier unit; Determining whether there is a vehicle in the real-time captured image; When a vehicle exists in the real-time captured image, a preset license plate detection and recognition module (6) is controlled to detect the license plate of the mobile carrier unit to identify the license plate information, and the license plate information is reported.

6. The intelligent mobile vehicle inspection method based on Beidou positioning and visual SLAM according to claim 5 is characterized in that: The step of controlling the preset license plate detection and recognition module (6) to detect and recognize the license plate of the mobile carrier unit comprises: Recognize the license plate image information based on the real-time captured image information of the mobile carrier unit; generating corrected license plate image information based on the license plate image information, and identifying the color of the license plate based on the license plate image information, thereby acquiring mobile carrier unit type information; Generate license plate text result information based on the corrected license plate image information; Output the license plate text result information and the mobile carrier unit type information.

Images