Intelligent pile searching method and system for electric vehicle based on machine vision

Abstract

The invention relates to the technical field of visual intelligent pile searching, and discloses an electric vehicle intelligent pile searching method and system based on machine vision, and the method comprises the steps: collecting the data of a parking lot in real time through a camera and a sensor, and transmitting the data to an intelligent algorithm server through an intelligent gateway; collected image video data are preprocessed through a data cleaning module, the processed data are analyzed and processed through an intelligent algorithm server, coordinate calibration and fusion are conducted on radar data and image data, and the intelligent algorithm server monitors the use state of the charging pile in real time. The state information is updated to the intelligent charging management platform in time; a user interacts with the intelligent charging management platform through the user terminal, and the method can quickly and accurately identify the position and state of the charging pile in the parking lot, provides real-time navigation guidance for the user, reduces the time for the user to search for the charging pile, and improves the use efficiency and management intelligence level of the charging pile.

Description

Technical Field

[0001] The present invention relates to the technical field of visual intelligent pile-finding technology, and more specifically to a method and system for intelligent pile-finding for electric vehicles based on machine vision. Background Art

[0002] With the rapid adoption of electric vehicles, charging piles, as critical infrastructure, are becoming increasingly important for their construction and management. However, current charging pile systems face significant challenges and shortcomings. For example, charging pile addressing is inefficient. Users typically rely on mobile apps or physical parking lot signs to find charging piles, which is inefficient in large parking lots or complex urban environments. Charging pile location identification is uncontrollable in both time and space, making it difficult for users to quickly identify available charging piles, particularly in crowded or poorly laid out parking lots. This leads to a poor user experience. Existing technologies cannot accurately monitor the real-time occupancy status of charging piles. Users often encounter situations where charging piles are occupied but still appear available. This information asymmetry increases search time and costs, leading to negative feedback. Charging pile fault status is not promptly updated and communicated to users and operators, impacting user experience and management efficiency. A lack of intelligent scheduling and management tools results in inefficient charging pile utilization. Resources cannot be flexibly allocated across different time periods and locations. Insufficient intelligent methods for charging pile maintenance and fault diagnosis result in delayed repair responses, undermining user trust. Furthermore, in complex parking lot routing and lighting conditions, the identification and provision of charging pile information present reliability issues, hindering the timely and accurate transmission of information.

[0003] To address these shortcomings, there is an urgent need to introduce more advanced technologies to improve the accuracy of charging pile location identification, the real-time status monitoring, and the intelligent level of management, so as to improve the user experience of charging piles, increase user satisfaction, and optimize resource management. Summary of the Invention

[0004] In order to overcome the above-mentioned defects of the prior art, the present invention provides an intelligent charging pile search method and system for electric vehicles based on machine vision. By introducing machine vision technology and intelligent algorithms, the utilization efficiency and management intelligence level of charging piles are improved, and the deficiencies of existing electric vehicle charging piles in search, usage status monitoring and management are solved, so as to solve the problems existing in the above-mentioned background technology.

[0005] The present invention provides the following technical solution: an intelligent pile-finding method for electric vehicles based on machine vision, comprising the following steps:

[0006] Step 1: Data collection and transmission: Deploy cameras, smart gateways, and environmental sensors at key locations in the parking lot to collect real-time image data, radar data, video information data, and environmental parameter data from the parking lot, and transmit the data to the intelligent algorithm server through the smart gateway;

[0007] Step 2: Data preprocessing and feature extraction: Preprocess the collected image data and video information data, analyze and process the preprocessed data based on the intelligent algorithm server, use the ORB algorithm to extract feature points from the image, and perform depth estimation on the extracted feature points;

[0008] Step 3: Multi-information fusion and 3D reconstruction: The radar data and image data are calibrated and fused. The image feature points extracted in step 2 are then fused with the radar data to form a dense point cloud and mapped into 3D space. The point cloud data is quickly triangulated and 3D reconstructed.

[0009] Step 4: Status Monitoring and Intelligent Analysis: The intelligent algorithm server monitors the usage status of charging piles in real time and promptly updates the status information to the intelligent charging management platform. The usage data of charging piles is stored and analyzed, and charging demand is predicted through data analysis. Intelligent scheduling is then performed based on the prediction results.

[0010] Step 5: User interaction and service: Users interact with the intelligent charging management platform through user terminals, and early warning information is sent to users and managers in a timely manner.

[0011] Preferably, the image data includes images of charging piles, vehicle images and personnel images; the radar data includes the location of charging piles, the distance between charging piles and the speed of vehicles; the video information data is the video image in the parking lot; the environmental parameters include the parking lot environment brightness, ambient temperature and ambient humidity.

[0012] Preferably, the specific steps of performing rapid triangulation processing on the point cloud data and performing three-dimensional reconstruction include:

[0013] Constructing a super tetrahedron in the three-dimensional space where the point cloud data is located, selecting four non-coplanar points from the point cloud as vertices of the super tetrahedron, and obtaining a circumscribed sphere of the tetrahedron formed by the four points;

[0014] Process each point in the point cloud, insert the points one by one, insert each point into the super tetrahedron, and for each inserted point, find the smallest tetrahedron that contains the point;

[0015] For each minimal tetrahedron, connect the vertices to form a new tetrahedron;

[0016] Make a circumscribed sphere for each newly formed tetrahedron. If the sphere does not contain any other points except the insertion point, the tetrahedron is considered to conform to the Delaunay triangulation. If it contains other points, perform edge flipping until all new tetrahedrons satisfy local Delaunay.

[0017] Repeat the above steps by inserting the next point, and make the inserted point and the generated tetrahedron satisfy the local Delaunay until all the point clouds are added to the super tetrahedron;

[0018] Eliminate the vertices of the super tetrahedron and all internal points, retaining only the surface triangle mesh, and optimize the triangle mesh to remove duplicate triangles and edges;

[0019] The final triangular mesh is smoothed to complete the 3D reconstruction process.

[0020] Preferably, for each minimum tetrahedron, the specific method of connecting vertices to form a new tetrahedron is:

[0021] Connect the new point to the four vertices of the smallest tetrahedron to form four new tetrahedrons;

[0022] Extract triangles from each newly formed tetrahedron and add them to the triangle mesh;

[0023] Delete the original minimal tetrahedron and form a new tetrahedron.

[0024] Preferably, the use of the ORB algorithm to extract feature points from an image and perform depth estimation on the extracted feature points is specifically as follows:

[0025] Based on the preprocessed data, the ORB algorithm is used to detect corner points in the image. The corner points serve as the basis of ORB feature points. For each detected corner point, a direction histogram is obtained to determine the main direction. A descriptor is generated for each corner point. The descriptor is a BRIEF feature point descriptor.

[0026] Use matching algorithms to match ORB feature points from multiple perspectives, identify corresponding feature points in the same scene, and form matching feature point pairs;

[0027] Based on the matched feature point pairs, the disparity of the corresponding pixel points of the feature points is obtained, the depth information of each feature point is estimated, and the two-dimensional image feature points are converted into points in three-dimensional space through the disparity information.

[0028] Preferably, the specific method of predicting charging demand through data analysis is:

[0029] The usage data of the charging pile includes the time of use of the charging pile, the location of the charging pile, the charging duration of each use, the charging power of each use, and the SOC of each charging pile battery;

[0030] Clean the charging pile usage data and use kernel density estimation to generate the regional charging demand distribution λ(x,y);

[0031] Construct a spatiotemporal graph convolutional network layer and an LSTM layer. The convolution kernel size of the spatiotemporal graph convolutional network layer is selected as 3; the time step of the LSTM layer is set to 24, and the number of hidden units is set to 64;

[0032] The usage data of charging piles are formed into a time series, and the generated regional charging demand distribution is used as the input of the model, and the charging request amount of the charging piles in the future time t is used as the model output.

[0033] Preferably, the regional charging demand distribution λ(x,y) is expressed as follows:

[0034] Among them, λ(x,y) represents the regional charging demand distribution of the charging pile at the location coordinate (x,y); h represents the domain parameter, that is, the domain range of the charging pile at the location coordinate (x,y), (x i ,y i ) represents the position coordinates of the i-th charging pile within the range of the charging pile with position coordinates at (x, y), n represents the number of charging piles within the range of the charging pile with position coordinates at (x, y), and K represents the Gaussian kernel function; i = 1, 2, 3, ..., n.

[0035] Preferably, the spatiotemporal graph convolutional network layer performs spatiotemporal graph convolution operation, the number of input channels is set to 16, and the number of output channels is set to 32; each charging pile is regarded as a node, the node feature is the historical charging amount sequence, and a graph adjacency matrix is ​​generated; the graph adjacency matrix is ​​expressed by the formula:

[0036] Among them, A ij represents the graph adjacency matrix between the i-th charging pile and the j-th charging pile within the domain; (x j ,y j ) represents the position coordinates of the jth charging pile within the range of the charging pile at the position coordinates (x, y); d th Represents the charging pile location distance threshold; j = 1, 2, 3, ..., n.

[0037] Preferably, the specific method of performing intelligent scheduling according to the prediction results is:

[0038] Obtaining all scheduling solutions that meet the predicted charging request amount based on the predicted charging request amount;

[0039] Select a scheduling solution that satisfies the objective function; the objective function is expressed as:

[0040]

[0041] Among them, Y is the objective function, min is the minimum value, ω brepresents the priority weight of the b-th vehicle, represents the time that the bth vehicle waits for charging under the cth scheduling scheme, B represents the total number of vehicles waiting to be charged, and P grid (c) represents the grid load under the charging time corresponding to the c-th scheduling scheme; max represents the maximum value, and β represents the grid load penalty coefficient.

[0042] An electric vehicle intelligent pile-finding system based on machine vision includes a data acquisition layer, a data transmission module, a data processing layer, and an application service layer;

[0043] The data acquisition layer is used to collect image data, radar data, video information data and environmental parameter data in the parking lot;

[0044] The data transmission module is used to ensure real-time and stable data transmission between the data acquisition layer and the data processing layer; it includes wired network equipment and wireless network equipment;

[0045] The data processing layer is mainly composed of an intelligent algorithm server and a data cleaning module; the intelligent algorithm server performs real-time analysis on the collected image data and video information data; the data cleaning module is used to pre-process the collected image and video data;

[0046] The application service layer is the interface between the system and the user, and mainly includes the intelligent charging management platform and the user terminal; the intelligent charging management platform is deployed on the cloud server and is responsible for storing and analyzing the usage data of the charging piles.

[0047] The technical effects and advantages of the present invention are as follows:

[0048] (1) Improved charging pile search efficiency: Through machine vision technology, the system can quickly and accurately identify the location and status of charging piles in the parking lot, providing users with real-time navigation guidance and reducing the time users spend searching for charging piles. The intelligent charging management platform stores and analyzes charging pile usage data, providing data support for users, facilitating the statistics and analysis of charging needs, achieving intelligent scheduling, and improving the scientificity and accuracy of management.

[0049] (2) Optimize resource management: The intelligent algorithm server monitors the usage status of charging piles in real time, determines whether they are occupied or working normally, and promptly updates the status information to the intelligent charging management platform, making it easier for managers to monitor and manage, thereby improving the efficiency of charging pile usage. The system can automatically allocate parking spaces based on the real-time status of the charging piles and vehicle demand, achieving intelligent control and improving charging efficiency and user experience.

[0050] (3) Improved user experience: Users interact with the intelligent charging management platform through user terminals such as smartphones and vehicle-mounted systems to obtain charging station information, make charging reservations, and receive navigation guidance services. This simplifies the charging process and improves charging convenience and satisfaction. Multi-information fusion technology combines radar detection data with visual image data to improve detection accuracy and robustness, enabling the system to operate stably under different environmental conditions.

[0051] (4) Enhanced security: The system effectively prevents unauthorized vehicles from entering the charging pile parking space through vehicle identification and identity verification, ensuring charging safety and orderly parking space use. At the same time, the early warning module sends early warning information to users and managers in a timely manner when the charging pile fails or an abnormal situation occurs, ensuring the safety and reliability of the charging process. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is a flow chart of the intelligent pile-finding method for electric vehicles based on machine vision of the present invention.

[0053] Figure 2 This is an architecture diagram of the electric vehicle intelligent pile-finding system based on machine vision of the present invention.

[0054] Figure 3 This is a flowchart for pile navigation.

[0055] Figure 4 This is a flow chart for monitoring charging pile usage.

[0056] Figure 5 This is a flowchart of intelligent scheduling and management. DETAILED DESCRIPTION

[0057] The technical solutions of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the present invention. In addition, the forms of the various structures described in the following embodiments are merely examples. The machine vision-based intelligent pile-finding method and system for electric vehicles involved in the present invention are not limited to the various structures described in the following embodiments. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0058] like Figure 1 As shown, the present invention provides an intelligent charging pile search method for electric vehicles based on machine vision. By introducing image processing technology and intelligent algorithms, real-time monitoring and analysis of charging piles and vehicles in a parking lot are achieved, providing users with accurate charging pile location information and navigation guidance, improving the efficiency of charging pile use and the level of intelligent management, and effectively solving the shortcomings of existing electric vehicle charging piles in search, usage status monitoring, and management. The method includes the following steps:

[0059] Step 1: Data Collection and Transmission: Deploy cameras, smart gateways, and environmental sensors at key locations in the parking lot to collect real-time image data, radar data, video information data, and environmental parameter data from the parking lot, and transmit the data to the intelligent algorithm server via the smart gateway. The image data includes, but is not limited to, images of charging piles, vehicles, and people; the radar data includes, but is not limited to, the location of charging piles, the distance between charging piles, and vehicle speed; the video information data is the video image of the parking lot; and the environmental parameters include, but are not limited to, the parking lot brightness, temperature, and humidity.

[0060] Step 2: Data preprocessing and feature extraction: The collected image data and video information data are preprocessed. The preprocessing includes but is not limited to cleaning, noise reduction, grayscale conversion, and scale normalization of the data to improve data quality, reduce noise in the image, and improve the accuracy of feature point detection, laying a data foundation for subsequent feature point extraction. Based on the intelligent algorithm server, machine vision algorithms and deep learning models are used to analyze and process the preprocessed data, and the ORB algorithm is used to extract feature points from the image. The depth of the extracted feature points is estimated to thicken the point cloud.

[0061] Step 3: Multi-information fusion and 3D reconstruction: The radar data and image data are calibrated and fused. The radar data is then cleaned twice. The image feature points extracted in step 2 are then fused with the radar data to form a dense point cloud and mapped into 3D space. The point cloud data is quickly triangulated. By constructing a super tetrahedron, inserting points, connecting vertices, and determining the circumscribed sphere, redundant points are removed, retaining the surface triangular mesh, and completing the 3D reconstruction.

[0062] Step 4: Status Monitoring and Intelligent Analysis: The intelligent algorithm server monitors the usage status of charging piles in real time, determines whether the charging piles are occupied and working properly, and promptly updates the status information to the intelligent charging management platform. The intelligent charging management platform stores and analyzes the usage data of charging piles, predicts charging demand through data analysis, and performs intelligent scheduling based on the prediction results.

[0063] Step 5: User interaction and service: Users interact with the intelligent charging management platform through user terminals to obtain charging pile information, make charging reservations, and obtain navigation guidance services. When a charging pile fails or experiences an abnormality, the intelligent charging management platform promptly sends warning information to users and managers.

[0064] In this embodiment, it should be specifically explained that, in the multi-information fusion and 3D reconstruction, the radar data and the image data are first calibrated to obtain radar data and image data with unified coordinates, and then the radar data and the image data with unified coordinates are fused to obtain fused data.

[0065] The radar data is cleaned twice, including fast Fourier transform, dynamic threshold segmentation processing, clustering processing and eliminating data that is more than a preset cluster radius from the cluster center, to obtain cleaned radar data;

[0066] The image feature points extracted in step 2 are fused with the radar data to form a dense point cloud, and mapped to a three-dimensional space, so that the feature points with depth information are mapped to the three-dimensional point cloud space, completing the first fusion; the point cloud data is quickly triangulated to obtain a three-dimensional plane; the specific steps include:

[0067] Constructing a super tetrahedron in the three-dimensional space where the point cloud data is located, selecting four non-coplanar points from the point cloud as vertices of the super tetrahedron, wherein the four points cover the entire point cloud space to the maximum extent, and obtaining a circumscribed sphere of the tetrahedron formed by the four points, so as to maximize the inclusion of all point cloud data;

[0068] Process each point in the point cloud, insert the points one by one, insert each point into the super tetrahedron, and for each inserted point, find the smallest tetrahedron that contains the point;

[0069] For each minimal tetrahedron, connect the vertices to form a new tetrahedron;

[0070] Make a circumscribed sphere for each newly formed tetrahedron. If the sphere does not contain any other points except the insertion point, the tetrahedron is considered to conform to the Delaunay triangulation. If it contains other points, perform edge flipping until all new tetrahedrons satisfy local Delaunay.

[0071] Repeat the above steps by inserting the next point, and make the inserted point and the generated tetrahedron satisfy the local Delaunay until all the point clouds are added to the super tetrahedron;

[0072] Eliminating vertices of the super tetrahedron and all internal points to retain only the surface triangle mesh, optimizing the triangle mesh to remove duplicate triangles and edges, and using a mesh optimization algorithm such as Poisson reconstruction or mesh simplification to improve the quality of the mesh and reduce the complexity of the mesh;

[0073] The final triangular mesh is smoothed to reduce the noise and irregularity of the mesh and complete the 3D reconstruction process.

[0074] In this embodiment, it should be specifically explained that, for each minimum tetrahedron, the specific method of connecting the vertices to form a new tetrahedron is:

[0075] Connect the new point to the four vertices of the smallest tetrahedron to form four new tetrahedrons;

[0076] Extract triangles from each newly formed tetrahedron and add them to the triangle mesh;

[0077] Delete the original minimal tetrahedron and form a new tetrahedron.

[0078] In this embodiment, it should be specifically explained that the use of the ORB algorithm to extract feature points from an image and perform depth estimation on the extracted feature points is specifically as follows:

[0079] Based on the preprocessed data, the ORB algorithm is used to detect corner points in the image. The corner points serve as the basis of ORB feature points. For each detected corner point, a direction histogram is obtained to determine the main direction, so that the ORB feature has rotation invariance. A descriptor is generated for each corner point. The descriptor is a BRIEF feature point descriptor, which is a fast binary descriptor.

[0080] Use matching algorithms such as the FLANN nearest neighbor search algorithm or the BFMatcher brute force matching algorithm to match ORB feature points from multiple perspectives to identify corresponding feature points in the same scene and form matching feature point pairs;

[0081] Based on the matched feature point pairs, the disparity of the feature points corresponding to the pixels is obtained to estimate the depth information of each feature point, and the two-dimensional image feature points are converted into points in three-dimensional space through the disparity information.

[0082] In this embodiment, it should be specifically explained that the specific method of predicting charging demand through data analysis is:

[0083] The usage data of the charging pile includes the time of use of the charging pile, the location of the charging pile, the charging duration of each use, the charging power of each use, and the SOC of each charging pile battery;

[0084] Perform data cleaning on the usage data of the charging piles, including but not limited to outlier removal and missing value filling; the missing value filling can use the spatiotemporal correlation interpolation method;

[0085] Generate regional charging demand distribution λ(x,y) using kernel density estimation;

[0086] Construct a spatiotemporal graph convolutional network layer and an LSTM layer. The convolution kernel size of the spatiotemporal graph convolutional network layer is selected as 3 to capture local spatial dependencies. The time step of the LSTM layer is set to 24, and the number of hidden units is set to 64 to balance computational complexity and feature capacity.

[0087] The usage data of charging piles are formed into a time series, and the generated regional charging demand distribution is used as the input of the model, and the charging request amount of the charging piles in the future time t is used as the model output.

[0088] In this embodiment, it should be specifically explained that the regional charging demand distribution λ(x, y) is expressed as follows:

[0089] Among them, λ(x,y) represents the regional charging demand distribution of the charging pile at the location coordinate (x,y); h represents the domain parameter, that is, the domain range of the charging pile at the location coordinate (x,y), (x i , y i) represents the position coordinates of the i-th charging pile within the range of the charging pile with the position coordinates at (c, y), n represents the number of charging piles within the range of the charging pile with the position coordinates at (x, y), and K represents the Gaussian kernel function; the values ​​of the field parameters can be set by those skilled in the art. If it is necessary to obtain the regional charging demand distribution within 500m of the charging pile with the position coordinates at (x, y), then h = 500; i = 1, 2, 3, ..., n.

[0090] In this embodiment, it should be specifically explained that the spatiotemporal graph convolutional network layer captures the spatial dependency between charging piles through a graph convolutional network (GCN) to model the dynamic correlation of charging demand between regions; each charging pile is regarded as a node, and the node feature is a historical charging amount sequence to generate a graph adjacency matrix; the spatiotemporal graph convolutional network layer performs a spatiotemporal graph convolution operation, the number of input channels is set to 16 to match the node feature dimension, and the number of output channels is set to 32 to improve the feature expression capability; the LSTM layer includes an input gate, a forget gate, and an output gate; the input gate is used to input the data output by the spatiotemporal graph convolutional network layer, and the output gate is used to output the charging request amount within the future time t;

[0091] The graph adjacency matrix is ​​expressed as follows:

[0092] Among them, A ij represents the graph adjacency matrix between the i-th charging pile and the j-th charging pile within the domain; (x j ,y j ) represents the position coordinates of the jth charging pile within the range of the charging pile at the position coordinates (x, y); d thIndicates the distance threshold of the charging pile location. In this embodiment, d th =1km, used to ensure the correlation of charging behavior in a local area. If the charging pile location distance threshold is too large, smoothing will result, and if the charging pile location distance threshold is too small, spatial information will be lost. The location coordinates of the charging piles can be GPS coordinates; j=1, 2, 3, ..., n.

[0093] In this embodiment, it should be specifically explained that the specific method of performing intelligent scheduling based on the prediction results is:

[0094] Based on the predicted charging request amount, all scheduling plans that meet the predicted charging request amount are obtained; the scheduling plan can be a collection of charging pile usage data such as charging time and charging power of all vehicles and charging piles, and the corresponding charging time of the vehicles and charging piles is represented by the correspondence between the vehicles to be charged and the charging time of the charging piles, for example, A:(x,y):(t start ,t finall ) represents the vehicle A corresponding to the charging pile with the position coordinate (x, y), and the charging time is (t start ,t finwall ), that is, vehicle A is charging at the charging pile with the coordinates (x, y), starting from t start Start charging, until t finall Charging is completed; the t start Indicates the time point when charging starts, t finall Indicates the time point when charging ends; you can add charging power and other charging pile usage data, for example, A:(x,y):(t start ,t finall ):P A , which means that vehicle A corresponds to the charging pile with the position coordinate (x, y), and the charging time is (t start ,t finall ), the charging power is P A ;

[0095] Select a scheduling solution that satisfies the objective function; the objective function is expressed as:

[0096]

[0097] Among them, Y is the objective function, min is the minimum value, ω b represents the priority weight of the b-th vehicle, represents the time that the bth vehicle waits for charging under the cth scheduling scheme, B represents the total number of vehicles waiting to be charged, and P grid(c) represents the grid load under the charging time corresponding to the c-th scheduling scheme; max represents the maximum value, β represents the grid load penalty coefficient, which can be determined by those skilled in the art based on the electricity price time-sharing policy. In this embodiment, β=0.1 is selected; the priority weight can be set according to the type of different vehicles. If the vehicle is an emergency vehicle such as a medical vehicle, police car, fire truck, etc., the priority weight can be set to 2. If the vehicle is an ordinary vehicle such as a household vehicle, the priority weight can be set to 1.

[0098] like Figure 2 As shown, the present invention provides an electric vehicle intelligent charging pile search system based on machine vision, including a data acquisition layer, a data transmission module, a data processing layer, and an application service layer. Through mutual cooperation, it effectively solves the problems of finding electric vehicle charging piles, monitoring and managing their usage status, and provides strong support for promoting the development of new energy electric vehicles. It can also achieve interconnection with a wider range of information systems such as urban traffic management systems, jointly promoting the progress of urban intelligence and sustainable development.

[0099] The data acquisition layer is used to collect image data, radar data, video information data and environmental parameter data in the parking lot, and mainly includes multiple cameras, radars, smart gateways and environmental sensors; the camera is one of the core components of the data acquisition layer and is deployed at key locations in the parking lot to ensure that all areas of the parking lot, especially the locations of each charging pile, can be fully covered; the cameras have different types and functions to adapt to diverse environmental conditions and acquisition needs, and the cameras include but are not limited to high-definition cameras, infrared cameras and wide-angle cameras; the high-definition camera is used to obtain high-resolution images and videos to more clearly capture details in the parking lot, such as the license plate number and model of the vehicle; the infrared camera is used to effectively capture images in low-light or no-light environments to ensure normal operation at night or in low-light conditions; the wide-angle camera is used to provide a wider viewing angle. The wide-angle camera can cover a larger area, reduce the number of cameras deployed, and ensure comprehensive monitoring of the parking lot; the cameras work together to collect image data and video information data in the parking lot in real time and transmit them to the smart gateway;

[0100] The intelligent gateway is a bridge between the data acquisition layer and the data transmission module, and undertakes the important task of data aggregation and forwarding. It receives data from each camera and performs preliminary processing on it, such as data compression and format conversion, to improve the efficiency and quality of data transmission. The processed data is transmitted to the intelligent algorithm server of the data processing layer via a wired or wireless network.

[0101] The environmental sensors are used to monitor environmental parameters in the parking lot, such as light intensity, temperature, and humidity. The data collected by the environmental sensors provides more comprehensive operating environment information, helps to better adapt to different environmental conditions, and provides a reference basis for subsequent data analysis and processing.

[0102] The data transmission module is used to ensure real-time and stable data transmission between the data acquisition layer and the data processing layer; it includes wired network equipment and wireless network equipment, such as routers, switches and wireless access points; the wired network equipment is used to connect the intelligent gateway and the intelligent algorithm server in the parking lot through wired media such as network cables or optical fibers to achieve high-speed and stable data transmission. The wired network equipment provides a reliable communication link to ensure the rapid transmission and processing of large amounts of data; the wireless network equipment is used to meet the access needs of mobile devices or temporarily deployed devices, and can allow user terminals and other devices to be wirelessly connected to the system through wireless communication technologies such as Wi-Fi, 4G / 5G, etc., so that users can obtain charging pile information and pile search services anytime and anywhere;

[0103] The data processing layer is mainly composed of an intelligent algorithm server and a data cleaning module, which is used to conduct in-depth analysis and processing of the collected data, extract valuable information, and provide support for the decision-making and operation of the system. The intelligent algorithm server runs advanced machine vision algorithms and deep learning models to perform real-time analysis of collected image data and video information data.

[0104] The main functions of the intelligent algorithm server include:

[0105] 1. Charging pile identification: Through the trained model, the intelligent algorithm server can accurately identify charging piles of different models and brands and obtain their location information;

[0106] 2. Vehicle detection: The system can identify vehicles in the parking lot, determine whether the vehicle is parked near a charging station, and whether it is connected to the charging station for charging;

[0107] 3. Status judgment: Based on the appearance changes of the charging pile and the connection status of the vehicle, the intelligent algorithm server determines whether the charging pile is occupied, whether it is working properly, and other status information;

[0108] 4. Feature point extraction and depth estimation: Using feature point extraction methods such as the ORB algorithm, key points in the image are extracted and depth estimation is performed in combination with radar data to obtain richer three-dimensional information. The charging pile radar can monitor the surrounding conditions of the charging pile in real time, and the fusion of this data with image data improves detection accuracy and better implements the pile-finding guidance function.

[0109] The data cleaning module is used to pre-process the collected image and video data, remove noise, correct distortion, etc., to improve data quality and provide a clearer and more accurate data basis for subsequent algorithm processing;

[0110] The application service layer is the interface between the system and the user, and mainly includes the intelligent charging management platform and user terminals. The application service layer converts the results of the data processing layer into services that users can understand and use, and provides management and monitoring functions for managers.

[0111] The intelligent charging management platform is deployed on a cloud server and is responsible for storing and analyzing charging pile usage data. Its main functions include:

[0112] 1. User Services: Provide users with services such as charging pile information query, charging reservation, and navigation guidance. Users can view the location, status, and other information of charging piles in real time through the platform and perform corresponding operations;

[0113] 2. Data analysis: The platform analyzes historical and real-time data of charging piles to predict charging demand, optimize resource allocation, and improve system operating efficiency;

[0114] 3. Monitoring and Management: Provides management personnel with functions such as monitoring charging pile status, dispatching resources, and maintaining equipment, facilitating effective management and maintenance of the entire system;

[0115] The user terminals include devices such as smartphones, vehicle-mounted systems and tablet computers; users interact with the intelligent charging management platform through these devices to obtain charging pile information, make charging reservations and receive navigation guidance and other services; the application design of the user terminals focuses on user experience, with a friendly interface and simple operation, ensuring that users can quickly and conveniently use the various functions of the system.

[0116] like Figure 3 As shown, the present invention provides a charging pile navigation process using the above-mentioned electric vehicle intelligent charging pile search method based on machine vision. When a user needs to find a charging pile, the system's workflow is as follows:

[0117] User-initiated request: When a user opens the app on their smartphone or vehicle system, the system automatically obtains the user's current location information and sends a charging station search request to the smart charging management platform;

[0118] Data processing and analysis: Based on user requests, the platform selects eligible charging piles from the database and sends their location information to the intelligent algorithm server. The server then verifies and updates the status of the charging piles based on real-time image and video data.

[0119] Navigation guide generation: The intelligent algorithm server plans the optimal navigation route based on the updated charging pile status information and sends the navigation guide information back to the user terminal. The user follows the navigation guide to quickly and accurately find an available charging pile.

[0120] like Figure 4 As shown, the present invention provides a charging pile usage monitoring process using the above-mentioned electric vehicle intelligent pile search method based on machine vision. During the user's charging process, the usage status of the charging pile is continuously monitored, specifically:

[0121] Real-time data collection: The camera continuously monitors the charging pile area, and the smart gateway transmits the real-time video stream to the intelligent algorithm server.

[0122] Status Analysis and Judgment: The server analyzes the video stream in real time to monitor the vehicle's charging status and the operation of the charging pile. If any abnormality is detected, such as a vehicle occupying the charging pile for an extended period without charging or a charging pile malfunctioning, the system will promptly generate an early warning.

[0123] Warning and notification: Warning information is sent to user terminals and managers' devices through the intelligent charging management platform, reminding users and managers to take corresponding measures, such as users ending charging and managers performing maintenance.

[0124] like Figure 5 As shown, the present invention provides an intelligent scheduling and management process using the above-mentioned machine vision-based electric vehicle intelligent charging pile search method. The intelligent charging management platform realizes intelligent scheduling and efficient management of charging pile resources based on data analysis results, specifically:

[0125] Data analysis and prediction: The platform analyzes historical and real-time data to explore users’ charging habits and demand patterns and predict future charging needs.

[0126] Resource scheduling and optimization: During peak hours, the platform optimizes resource allocation based on forecast results by adjusting charging power and guiding users to other charging stations to ensure the efficient operation of charging services.

[0127] Violation management and handling: For vehicles that illegally occupy charging piles, the system will take corresponding measures after identification, such as sending reminder notifications, recording violations, etc., to improve the turnover rate and utilization efficiency of the charging piles.

[0128] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

[0129] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. An intelligent pile-finding method for electric vehicles based on machine vision, characterized by: The following steps are involved: Step 1: Data collection and transmission: Deploy cameras, smart gateways, and environmental sensors at key locations in the parking lot to collect real-time image data, radar data, video information data, and environmental parameter data from the parking lot, and transmit the data to the intelligent algorithm server through the smart gateway; Step 2: Data preprocessing and feature extraction: Preprocess the collected image data and video information data, analyze and process the preprocessed data based on the intelligent algorithm server, use the ORB algorithm to extract feature points from the image, and perform depth estimation on the extracted feature points; Step 3: Multi-information fusion and 3D reconstruction: The radar data and image data are calibrated and fused. The image feature points extracted in step 2 are then fused with the radar data to form a dense point cloud and mapped into 3D space. The point cloud data is quickly triangulated and 3D reconstructed. Step 4: Status Monitoring and Intelligent Analysis: The intelligent algorithm server monitors the usage status of charging piles in real time and promptly updates the status information to the intelligent charging management platform. The usage data of charging piles is stored and analyzed, and charging demand is predicted through data analysis. Intelligent scheduling is then performed based on the prediction results. Step 5: User interaction and service: Users interact with the intelligent charging management platform through user terminals, and early warning information is sent to users and managers in a timely manner.

2. The method for intelligently finding a pile for an electric vehicle based on machine vision according to claim 1, characterized in that: The image data includes images of charging piles, vehicles and people; the radar data includes the location of charging piles, the distance between charging piles and the speed of vehicles; the video information data is the video image in the parking lot; the environmental parameters include the brightness, temperature and humidity of the parking lot environment.

3. The method for intelligently finding a pile for an electric vehicle based on machine vision according to claim 2, characterized in that: The specific steps of performing rapid triangulation processing on point cloud data and performing three-dimensional reconstruction include: Constructing a super tetrahedron in the three-dimensional space where the point cloud data is located, selecting four non-coplanar points from the point cloud as vertices of the super tetrahedron, and obtaining a circumscribed sphere of the tetrahedron formed by the four points; Process each point in the point cloud, insert the points one by one, insert each point into the super tetrahedron, and for each inserted point, find the smallest tetrahedron that contains the point; For each minimal tetrahedron, connect the vertices to form a new tetrahedron; Make a circumscribed sphere for each newly formed tetrahedron. If the sphere does not contain any other points except the insertion point, the tetrahedron is considered to conform to the Delaunay triangulation. If it contains other points, perform edge flipping until all new tetrahedrons satisfy local Delaunay. Repeat the above steps by inserting the next point, and make the inserted point and the generated tetrahedron satisfy the local Delaunay until all the point clouds are added to the super tetrahedron; Eliminate the vertices of the super tetrahedron and all internal points, retaining only the surface triangle mesh, and optimize the triangle mesh to remove duplicate triangles and edges; The final triangular mesh is smoothed to complete the 3D reconstruction process.

4. The method for intelligently finding a pile for an electric vehicle based on machine vision according to claim 3, characterized in that: For each minimum tetrahedron, the specific method of connecting vertices to form a new tetrahedron is: Connect the new point to the four vertices of the smallest tetrahedron to form four new tetrahedrons; Extract triangles from each newly formed tetrahedron and add them to the triangle mesh; Delete the original minimal tetrahedron and form a new tetrahedron.

5. The method for intelligently finding a pile for an electric vehicle based on machine vision according to claim 4, characterized in that: The ORB algorithm is used to extract feature points from an image and perform depth estimation on the extracted feature points. Specifically: Based on the preprocessed data, the ORB algorithm is used to detect corner points in the image. The corner points serve as the basis of ORB feature points. For each detected corner point, a direction histogram is obtained to determine the main direction. A descriptor is generated for each corner point. The descriptor is a BRIEF feature point descriptor. Use matching algorithms to match ORB feature points from multiple perspectives, identify corresponding feature points in the same scene, and form matching feature point pairs; Based on the matched feature point pairs, the disparity of the corresponding pixel points of the feature points is obtained, the depth information of each feature point is estimated, and the two-dimensional image feature points are converted into points in three-dimensional space through the disparity information.

6. The method for intelligently finding a pile for an electric vehicle based on machine vision according to claim 5, characterized in that: The specific method of predicting charging demand through data analysis is: The usage data of the charging pile includes the time of use of the charging pile, the location of the charging pile, the charging duration of each use, the charging power of each use, and the SOC of each charging pile battery; Clean the charging pile usage data and use kernel density estimation to generate the regional charging demand distribution λ(x,y); Construct a spatiotemporal graph convolutional network layer and an LSTM layer. The convolution kernel size of the spatiotemporal graph convolutional network layer is selected as 3; the time step of the LSTM layer is set to 24, and the number of hidden units is set to 64; The usage data of charging piles are formed into a time series, and the generated regional charging demand distribution is used as the input of the model, and the charging request amount of the charging piles in the future time t is used as the model output.

7. The method for intelligently finding a pile for an electric vehicle based on machine vision according to claim 6, characterized in that: The regional charging demand distribution λ(x,y) is expressed as follows: Among them, λ(x,y) represents the regional charging demand distribution of the charging pile at the location coordinate (x,y); h represents the domain parameter, that is, the domain range of the charging pile at the location coordinate (x,y), (x i ,y i ) represents the position coordinates of the i-th charging pile within the range of the charging pile with position coordinates at (x, y), n represents the number of charging piles within the range of the charging pile with position coordinates at (x, y), and K represents the Gaussian kernel function; i = 1, 2, 3, ..., n.

8. The method for intelligently finding a pile for an electric vehicle based on machine vision according to claim 7, characterized in that: The spatiotemporal graph convolutional network layer performs spatiotemporal graph convolution operations, with the number of input channels set to 16 and the number of output channels set to 32. Each charging pile is regarded as a node, and the node feature is the historical charging amount sequence, generating a graph adjacency matrix. The graph adjacency matrix is ​​expressed as follows: Among them, A ij represents the graph adjacency matrix between the i-th charging pile and the j-th charging pile within the domain; (x j ,y j ) represents the position coordinates of the jth charging pile within the range of the charging pile at the position coordinates (x, y); d th Represents the charging pile location distance threshold; j = 1, 2, 3, ..., n.

9. The method for intelligently finding a pile for an electric vehicle based on machine vision according to claim 8, characterized in that: The specific method of performing intelligent scheduling based on the prediction results is: Obtaining all scheduling solutions that meet the predicted charging request amount based on the predicted charging request amount; Select a scheduling solution that satisfies the objective function; the objective function is expressed as: Among them, Y is the objective function, min is the minimum value, ω b represents the priority weight of the b-th vehicle, represents the time that the bth vehicle waits for charging under the cth scheduling scheme, B represents the total number of vehicles waiting to be charged, and P grid (c) represents the grid load under the charging time corresponding to the c-th scheduling scheme; max represents the maximum value, and β represents the grid load penalty coefficient.

10. A machine vision-based intelligent pile-finding system for electric vehicles, adapted to utilize the machine vision-based intelligent pile-finding method for electric vehicles according to any one of claims 1 to 9, characterized in that: Including data acquisition layer, data transmission module, data processing layer and application service layer; The data acquisition layer is used to collect image data, radar data, video information data and environmental parameter data in the parking lot; The data transmission module is used to ensure real-time and stable data transmission between the data acquisition layer and the data processing layer; it includes wired network equipment and wireless network equipment; The data processing layer is mainly composed of an intelligent algorithm server and a data cleaning module; the intelligent algorithm server performs real-time analysis on the collected image data and video information data; the data cleaning module is used to pre-process the collected image and video data; The application service layer is the interface between the system and the user, and mainly includes the intelligent charging management platform and the user terminal; the intelligent charging management platform is deployed on the cloud server and is responsible for storing and analyzing the usage data of the charging piles.

Images