Three-dimensional map generation method and system based on video recognition

Abstract

The invention relates to the technical field of image analysis, in particular to image data processing or generation. According to the technical scheme, the three-dimensional map generation method based on video recognition comprises the following steps that all modules are initialized, and pictures are captured in real time and preliminarily processed; performing three-dimensional rendering according to the specific view angle map picture of the instruction; preprocessing the rendered image of the three-dimensional map and the preliminarily processed real-time picture sum; calculating the similarity of the two processed images by using an image similarity algorithm; and generating a standard three-dimensional map rendering image, and storing and updating the standard three-dimensional map rendering image to a database. The three-dimensional map generation method and system based on video recognition solve the problems that in the prior art, the workload is large, errors are prone to occurring, maintenance is difficult and accuracy is poor, and the purposes of automatic matching, high accuracy, maintenance simplification and good stability are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of image analysis, and in particular to image data processing or generation. Background Art

[0002] Currently, the main method of converting video images into 3D maps is to establish a data table correspondence between the camera and the view images of each point on the 3D map to achieve automatic switching of the view angle. This method requires the operator to manually enter the view angle correspondence between each camera image and the corresponding point on the 3D map in advance.

[0003] For example, a Chinese patent with publication number CN109523579B discloses a method and device for matching drone video images with three-dimensional maps, and provides the following technical solutions: a method and device for matching drone video images with three-dimensional maps, which are used to calculate the pitch angle α, roll angle β and angle θ of the camera centerline relative to the vertical line of the ground, and calculate the coordinates of each vertex of the quadrilateral of the video image taken by the drone based on the above parameters; divide each side of the quadrilateral of the video image into three equal parts according to the field of view angle, and calculate the coordinates of the equally divided points of each side; interpolate the positions of the four points already obtained on each side to obtain the coordinates of the middle points between the adjacent points of the four points on each side; when the drone is at a high position and is in an open area or on the boundary of a low target area, connect the calculated middle points on each edge on the map to obtain the projection of the video boundary on the map; when the drone is flying at a low altitude and is blocked by tall objects, identify N tall target points in the drone video through an image recognition algorithm, and judge whether the N points are located in the vertex area or the boundary area of ​​the video area to distinguish and calculate the projection area. The above method realizes the accurate matching of the video image taken by the drone with the marked target on the three-dimensional map. However, the above method and device for matching the drone video image with the three-dimensional map is mainly achieved by finding tall target points and calculating distance coordinates. It is difficult to use when there is no suitable target point and it is difficult to calculate the distance coordinates. Summary of the invention

[0004] The present invention solves the problems of heavy workload, easy errors, difficult maintenance and poor accuracy in the prior art, and proposes a three-dimensional map generation method and system based on video recognition, which achieves the purposes of automated matching, high accuracy, simplified maintenance and good stability.

[0005] To achieve the above object, the present invention adopts the following technical solution: A method for generating a three-dimensional map based on video recognition comprises the following steps: S1: Initialize each module, capture the image in real time and perform preliminary processing on the image; S2: 3D rendering of the map screen at a specific viewing angle according to the command; S3: pre-processing the 3D map rendering image and the real-time image after preliminary processing; S4: using an image similarity algorithm to calculate the similarity of the two processed images; S5: Generate a qualified three-dimensional map rendering image and store the update in a database.

[0006] A three-dimensional map generation system based on video recognition includes: a three-dimensional map module and a control module, wherein the three-dimensional map module is electrically connected to an image processing unit, wherein the image processing unit is communicatively connected to a camera module, and the control module is electrically connected to the three-dimensional map module, a database module and an image recognition module, wherein the image recognition module is connected to the image processing unit.

[0007] The advantage of this design is that it can realize automatic 3D map generation through systematic steps, improve work efficiency and accuracy, reduce human errors, and enable efficient communication and collaboration between modules through the system, ensuring the overall performance and reliability of the system.

[0008] Preferably, the step S3 includes the following steps: S3.1: Acquire the original image to be processed, wherein the original image includes a three-dimensional map rendering image and a real-time image after preliminary processing; S3.2: resize the image to the desired size and convert the color image to grayscale image; S3.3: Use edge detection to extract image edge features; S3.4: Extract image feature points by establishing Gaussian pyramid and difference Gaussian pyramid; S3.5: Save or display the processed image and output the detected edge and feature point information.

[0009] The advantage of this design is that it can unify the processing steps, improve the standardization and consistency of image preprocessing, and ensure the accuracy of subsequent image matching.

[0010] Preferably, the step S3.3 includes the following steps: S3.31: Use Gaussian filter to smooth the image; S3.32: Calculate the gradient of the image in the x and y directions; S3.33: Perform non-maximum suppression on the gradient amplitude and retain the local maximum as a potential edge point; S3.34: Use high and low thresholds to classify the gradient amplitude and determine strong and weak edges; S3.35: Connect strong edges and weak edges through hysteresis thresholding to form a complete edge image.

[0011] The advantage of this design is that it improves the accuracy and robustness of image edge detection and ensures that the extracted edge information is more reliable and clear.

[0012] Preferably, the step S4 includes the following steps: S4.1: receiving two pre-processed images to be compared; S4.2: Extract features from the image, perform feature matching on the extracted features, and apply ratio test to select high-quality matches; S4.3: Perform geometric verification on the matching results to remove incorrect matches; S4.4: Calculate the similarity and determine whether they are similar according to the set threshold. If so, proceed to step S5; if not, proceed to step S4.5; S4.5: The control module instructs the three-dimensional map module to adjust the viewing angle according to a predetermined step length, and returns to step S2.

[0013] The advantage of this design is that it can ensure the accuracy and reliability of the matching results, thereby improving the accuracy of three-dimensional map generation.

[0014] Preferably, in step S3.2, adjusting the image size to the desired size includes adjusting the image size to the desired size using bilinear interpolation or nearest neighbor interpolation.

[0015] The advantage of this design is that the image scaling method can be flexibly selected to adapt to different application requirements and ensure the efficiency and effect of image processing.

[0016] Preferably, the step S3.4 includes the following steps: S3.41: Generate Gaussian pyramid and difference Gaussian pyramid; S3.42: Find potential feature points in scale space and accurately locate the feature points by interpolation; S3.43: Assign a main direction to each keypoint and generate a description vector based on the area around the keypoint.

[0017] The advantage of this design is that it can accurately extract the feature points of the image and improve the accuracy and reliability of feature matching.

[0018] Preferably, in step S1, the initialization of each module includes a control module loading initial parameters, and the initial parameters include a similarity threshold and a viewing angle adjustment step; the real-time capture of the picture and the preliminary processing of the picture include a camera module continuously capturing the real-time picture, transmitting the digital signal to the image processing unit through an image acquisition card, and the image processing unit performing preliminary processing on the received picture, and the preliminary processing includes denoising and adjusting contrast; in step S2, the three-dimensional rendering of the map picture of a specific viewing angle according to the instruction includes the three-dimensional map module rendering the map picture of a specific viewing angle according to the instruction of the control module, and transmitting it to the image processing unit after the rendering is completed.

[0019] Preferably, the step S5 includes: generating a qualified three-dimensional map rendering image, the control module sending the correspondence between the real-time picture and the three-dimensional map perspective to the database module for storage, the database module updating the stored correspondence, and returning to the step of capturing the picture in real time and performing preliminary processing on the picture in step S1; the relevant data includes a timestamp, a camera ID, and three-dimensional map perspective parameters.

[0020] The advantage of this design is that it can systematically store and update the data relationship of the three-dimensional map perspective, facilitate subsequent query and management, and improve the maintenance efficiency of the system.

[0021] Preferably, the camera module captures video images in real time, the image module preprocesses and calculates similarity of the image, the three-dimensional map module performs multi-perspective three-dimensional rendering of the map, the control module switches the perspective and calculates similarity, and the database module saves the similarity and the corresponding relationship after successful comparison.

[0022] The advantage of this design is that it can realize the automated operation of the system, ensure the real-time capture, processing and updating of the three-dimensional map perspective, and improve the overall efficiency and accuracy of the system.

[0023] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention realizes automatic 3D map generation through systematic steps, from initialization module to real-time screen capture, to image preprocessing and similarity calculation, each step is aimed at reducing manual intervention and improving processing speed and efficiency. This automated process not only reduces human errors, but also greatly improves work efficiency, making the generation of 3D maps faster and more accurate.

[0024] 2. The present invention ensures the standardization and consistency of image preprocessing by using advanced image processing techniques such as Gaussian filtering, edge detection and feature point extraction. These techniques improve the accuracy and robustness of image edge detection, ensuring that the extracted edge and feature point information is more reliable and clear. In addition, precise feature matching and geometric verification further enhance the accuracy of the matching results.

[0025] 3. The present invention is designed with a complete system architecture, including a camera module, an image processing unit, a three-dimensional map module, a control module and a database module, and the modules communicate and collaborate efficiently. This systematic design not only ensures real-time capture, processing and updating of the three-dimensional map perspective, but also simplifies the maintenance of the system, improves the flexibility and adaptability of the system, and makes the entire system more stable and easy to manage. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 The present invention is a flowchart of an embodiment of a method for generating a three-dimensional map based on video recognition.

[0027] Figure 2 The system block diagram of an embodiment of a three-dimensional map generation system based on video recognition in the present invention. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical scheme and advantages of the present disclosure clearer, the following will further describe the embodiments of the present disclosure in detail with reference to the accompanying drawings. The proportions of the components are not drawn according to the actual proportions, and the proportions and sizes shown in the accompanying drawings should not limit the substantial technical scheme of the present invention. These embodiments do not describe all the details in detail, nor do they limit the invention to the specific embodiments described.

[0029] See also Figure 1-2 As shown, a three-dimensional map generation method based on video recognition includes the following steps: S1: Initialize each module, capture the image in real time and perform preliminary processing on the image; S2: 3D rendering of the map screen at a specific viewing angle according to the command; S3: pre-processing the 3D map rendering image and the real-time image after preliminary processing; S4: using an image similarity algorithm to calculate the similarity of the two processed images; S5: Generate a qualified three-dimensional map rendering image and store the update in a database.

[0030] A three-dimensional map generation system based on video recognition includes: a three-dimensional map module and a control module, wherein the three-dimensional map module is electrically connected to an image processing unit, wherein the image processing unit is communicatively connected to a camera module, and the control module is electrically connected to the three-dimensional map module, a database module and an image recognition module, wherein the image recognition module is connected to the image processing unit.

[0031] like Figure 1 In one embodiment shown, Figure 1 The present invention is a flowchart of an embodiment of a method for generating a three-dimensional map based on video recognition.

[0032] Step S1: Initialization and real-time capture. First, in step S1, the system initializes each module and loads the initial parameters. These initial parameters include similarity threshold and viewing angle adjustment step size. After initialization, the camera module starts to continuously capture real-time images and transmits digital signals to the image processing unit through the image acquisition card. The image processing unit performs preliminary processing on the received images, including denoising and contrast adjustment. The benefit of this step is that by denoising and adjusting the contrast, the image quality of subsequent processing can be improved to ensure stable operation of the system.

[0033] Step S2: 3D rendering. Next, in step S2, according to the instruction of the control module, the 3D map module performs 3D rendering on the map screen of a specific viewing angle. After the rendering is completed, the rendered image is transmitted to the image processing unit. The benefit of 3D rendering is that it enables the system to generate 3D map images of different viewing angles as needed, providing rich data support for subsequent steps.

[0034] Step S3: Image preprocessing: In step S3, the system preprocesses the 3D map rendering image and the real-time image after preliminary processing. Specifically, it includes the following sub-steps: Step S3.1: First, the original images to be processed are obtained, which include three-dimensional map rendering images and real-time images after preliminary processing.

[0035] Step S3.2: Resize the image to the desired size and convert the color image to a grayscale image. This step can be done by bilinear interpolation or nearest neighbor interpolation. The benefits of resizing the image and color conversion are reduced computational complexity and increased processing speed.

[0036] Step S3.3: Use edge detection to extract image edge features. The specific process includes using a Gaussian filter to smooth the image, calculating the gradient of the image in the x and y directions, performing non-maximum suppression on the gradient amplitude, retaining the local maximum as a potential edge point, and determining strong edges and weak edges through high and low threshold classification, and finally connecting strong and weak edges through the hysteresis threshold method to form a complete edge image. The benefit of edge detection is that it helps to extract the structural information of the image and enhance the image features.

[0037] Step S3.4: Extract image feature points by building Gaussian pyramids and differential Gaussian pyramids. Find potential feature points in the scale space and accurately locate the feature point positions through interpolation. Then, assign a main direction to each key point and generate a description vector based on the area around the key point. The Gaussian pyramid method can effectively extract image features of different scales and enhance the robustness of feature points.

[0038] Step S3.5: Save or display the processed image and output the detected edge and feature point information.

[0039] Step S4: Image similarity calculation. In step S4, the system uses an image similarity algorithm to calculate the similarity of the two processed images. Specifically, it includes the following sub-steps: Step S4.1: receiving two pre-processed images to be compared.

[0040] Step S4.2: Extract features from the image and perform feature matching on the extracted features. Apply ratio test to select high-quality matches. The benefits of feature matching and ratio test are that they can improve matching accuracy and reduce false matches.

[0041] Step S4.3: Perform geometric verification on the matching results to remove false matches. Geometric verification can further improve the reliability of the matching results.

[0042] Step S4.4: Calculate the similarity and determine whether they are similar based on the set threshold. If they are similar, proceed to step S5; if they are not similar, proceed to step S4.5.

[0043] Step S4.5: The control module instructs the three-dimensional map module to adjust the viewing angle according to a predetermined step length, and returns to step S2. The advantage of this step is that by adjusting the viewing angle, different angles can be tried to find a more suitable matching result.

[0044] Step S5: Generate and update the three-dimensional map. Finally, in step S5, the system generates a qualified three-dimensional map rendering image and sends the correspondence between the real-time picture and the three-dimensional map perspective to the database module for storage. The database module updates the stored correspondence and returns to the step of capturing the picture in real time and performing preliminary processing on the picture in step S1. The relevant data includes timestamp, camera ID, and three-dimensional map perspective parameters. The benefit of this step is that the continuous updating and storage of the correspondence helps to build a more complete and accurate three-dimensional map.

[0045] The three-dimensional map generation method based on video recognition designed by the present invention realizes efficient conversion and continuous updating from video images to three-dimensional maps through a series of precise processing steps. The main advantages of this method include: Real-time and high efficiency: By capturing and processing video images in real time, the system can quickly generate and update three-dimensional maps to adapt to dynamically changing environments.

[0046] High precision: A variety of image processing technologies are used, such as edge detection and Gaussian pyramid, to ensure the accuracy of feature extraction and improve the accuracy of three-dimensional maps.

[0047] Robustness: Through steps such as feature matching and geometric verification, false matches are effectively reduced, and the robustness and reliability of the system are enhanced.

[0048] Flexibility: The system can adjust the viewing angle as needed and try different angles to find the best match, which has high flexibility.

[0049] Scalability: Through modular design, the system is easy to expand and upgrade, and can adapt to more application scenarios and more complex environments.

[0050] Overall, this method provides a comprehensive and efficient solution for 3D map generation based on video recognition, and has significant advantages in many aspects, and is suitable for many fields such as intelligent surveillance, autonomous driving, virtual reality, etc.

[0051] In one embodiment, the following scheme is adopted: (1) System initialization: When the system starts, all modules are initialized, including the camera module, image processing unit, image recognition module, control module, 3D map module, and database module. The control module loads initial parameters, such as similarity threshold, viewing angle adjustment step, etc.

[0052] (2) Real-time image capture: The camera module continuously captures real-time images and transmits digital signals to the image processing unit through the image acquisition card. The image processing unit performs preliminary processing on the received images, such as denoising and adjusting contrast.

[0053] (3) Image preprocessing: the image processing unit performs unified preprocessing on the real-time image and the three-dimensional map rendering image, including but not limited to: a) Image scaling: unify image size; b) Grayscale: convert color images into grayscale images; c) Edge detection: Use algorithms such as Canny or Sobel to extract image edge features; d) Feature point extraction: Use SIFT, SURF and other algorithms to extract image feature points.

[0054] (4) 3D map rendering: The 3D map module renders a map image of a specific perspective according to the instructions of the control module. The rendered image is transmitted to the image processing unit and pre-processed in the same way as the real-time image.

[0055] (5) Image similarity calculation: The image recognition module receives the pre-processed real-time image and the 3D map rendering image and uses a selected image similarity algorithm (such as the SIFT algorithm based on feature point matching or a deep learning image similarity network) to calculate the similarity of the two images.

[0056] (6) Similarity judgment and viewing angle adjustment: The control module receives the similarity calculation result from the image recognition module. If the similarity reaches a preset threshold, it is considered that the current 3D map viewing angle matches the real-time image. If the similarity does not meet the threshold, the control module instructs the 3D map module to adjust the viewing angle according to a predetermined step size and repeats steps (4)-(6).

[0057] (7) Result storage and update: When a matching view is found, the control module sends the correspondence between the real-time image and the 3D map view to the database module for storage. The database module updates the stored correspondence, including information such as timestamp, camera ID, and 3D map view parameters.

[0058] (8) Continuous monitoring and optimization: The system continuously executes steps (2)-(7), monitors the image changes in real time and updates the 3D map perspective. The control module can optimize the perspective search strategy based on historical matching data to improve matching efficiency.

[0059] Among them, the image preprocessing scheme is as follows: (1) Input image and obtain the original image to be processed.

[0060] (2) Image scaling: Use bilinear interpolation, nearest neighbor interpolation, etc. to adjust the image size to the required size.

[0061] (3) Grayscale: convert the color image into a grayscale image.

[0062] Formula: Gray=0.299R+0.587G+0.114B (3) Edge detection (Canny algorithm): Gaussian filtering: Use a Gaussian filter to smooth the image and reduce the impact of noise on edge detection.

[0063] Gaussian filter formula: Calculate gradient: Calculate the gradient of the image in the x and y directions using the Sobel operator.

[0064] Sobel operator: Gradient Amplitude: Gradient direction:

[0065] Non-maximum suppression: Perform non-maximum suppression on the gradient amplitude and retain the local maximum as potential edge points.

[0066] Dual threshold detection: Use high and low thresholds to classify the gradient amplitude and determine strong and weak edges.

[0067] Edge tracking: Connect strong edges and weak edges through the hysteresis threshold method to form a complete edge image.

[0068] (4) Feature point extraction (SIFT algorithm), constructing scale space: generating Gaussian pyramid and difference Gaussian pyramid.

[0069] Detecting local extrema: Finding potential feature points in scale space.

[0070] Accurately locate key points: Accurately locate feature points through interpolation.

[0071] Compute keypoint orientations: Assign a principal orientation to each keypoint.

[0072] Generate feature descriptors: Generate description vectors based on the areas around key points.

[0073] (5) Output the processing results, save or display the processed image, and output the detected edge and feature point information.

[0074] Among them, the image similarity calculation scheme is as follows: Image input, load two pre-processed image files to be compared.

[0075] Library used: OpenCV (cv2), code: import cv2 import numpy as np #Read two images image1=cv2.imread('image1.jpg',cv2.IMREAD_GRAYSCALE) image2=cv2.imread('image2.jpg',cv2.IMREAD_GRAYSCALE).

[0076] Feature matching, SIFT is used for feature extraction, then FLANN is used for feature matching, and the ratio test is applied to screen high-quality matches.

[0077] When using FLANN for feature matching, Lowe's ratio test is used to screen high-quality matches. Assuming d1 is the distance of the best match and d2 is the distance of the second best match, the conditions for the ratio test are: ratio_threshold is set to 0.7.

[0078] The relevant code is: #Create SIFT feature detector sift = cv2.SIFT_create() #Detect and calculate feature points and descriptors keypoints1,descriptors1=sift.detectAndCompute(image1,None) keypoints2,descriptors2=sift.detectAndCompute(image2,None) #Use FLANN for feature matching FLANN_INDEX_KDTREE=1 index_params=dict(algorithm=FLANN_INDEX_KDTREE,trees=5) search_params=dict(checks=50) flann=cv2.FlannBasedMatcher(index_params,search_params) matches=flann.knnMatch(descriptors1,descriptors2,k=2) # Apply ratio test to filter out high-quality matches ratio_threshold=0.7 good_matches=[m for m,n in matches if m.distance <ratio_threshold*n.distance]。

[0079] Geometric verification, uses the RANSAC algorithm to perform geometric verification and remove false matches.

[0080] Assume N inliers is the number of interior points, and N total is the total number of matching points, then the ratio of internal points is:

[0081] The relevant code is: #Geometry verification: using RANSAC algorithm if len(good_matches)>4: src_pts = np.float32([keypoints1[m.queryIdx].pt for min good_matches]).reshape(-1,1,2) dst_pts = np.float32([keypoints2[m.trainIdx].pt formin good_matches]).reshape(-1,1,2) # Use RANSAC algorithm to calculate the homography matrix H,mask=cv2.findHomography(src_pts,dst_pts,cv2.RANSAC,5.0) matches_mask=mask.ravel().tolist() # Calculate the number of inliers N_inliers=np.sum(matches_mask) else: matches_mask = [] N_inliers=0.

[0082] Similarity calculation, similarity score: When calculating similarity, the number of matching points and the proportion of RANSAC inliers are combined: Similarity based on the number of matching points: Similarity based on RANSAC inlier ratio: Comprehensive similarity score:

[0083] The relevant code is: # Calculate similarity score N_good_matches=len(good_matches) N_keypoints1=len(keypoints1) N_keypoints2=len(keypoints2) #Similarity based on the number of matching points similarity_matches=N_good_matches / min(N_keypoints1,N_keypoints2) #Similarity based on RANSAC internal point ratio similarity_inliers=N_inliers / N_good_matches if N_good_matches>0else0 #Comprehensive similarity score similarity_score=(similarity_matches+similarity_inliers) / 2.

[0084] Result evaluation, setting a threshold and judging the degree of similarity, by setting a similarity threshold to determine whether the images are similar:

[0085] Then output the similarity score and judgment result.

[0086] The relevant code is: #Set similarity threshold threshold=0.5 #Judge the similarity result = "similar" if similarity_score>threshold else "not similar" # Output similarity score and judgment result print(f"Similarity score:{similarity_score:.2f}") print(f"Judgement result:{result}").

[0087] like Figure 2 In one embodiment shown, Figure 2 The system block diagram of an embodiment of a three-dimensional map generation system based on video recognition of the present invention. The three-dimensional map generation system based on video recognition designed by the present invention mainly includes the following modules: (1) Camera module: responsible for capturing real-time video images; (2) Image recognition module: image preprocessing and similarity calculation; (3) 3D map module: provides multi-view rendering of maps; (4) Control module: controls the view switching and similarity calculation process; (5) Database module: saves the corresponding relationship after successful similarity comparison.

[0088] Among them, the camera module captures video images in real time, the image module preprocesses and calculates similarity of the image, the three-dimensional map module performs multi-perspective three-dimensional rendering of the map, the control module switches the perspective and calculates the similarity, and the database module saves the similarity and the corresponding relationship after successful comparison.

[0089] After the system identifies the imbalance problem, it enters steps S3 and S4 to adjust the signal settings to alleviate the imbalance. Step S3 involves obtaining the current signal control scheme and making phase adjustments. The system first obtains the current signal control scheme and analyzes the import with a larger flow rate among the imports opposite the imbalance alarm. Next, the system determines whether a separate phase has been set for the import with a larger flow rate among the imports opposite the imbalance alarm. If a separate phase has been set, the system directly enters step S4 to adjust the green light duration; if not, the system needs to set a separate phase for the left turn and straight flow directions of the import, and insert it between the straight phase and the left turn phase. This adjustment method ensures that imports with larger flow rates can get more traffic opportunities, thereby alleviating traffic pressure.

[0090] In step S4, the system first calculates the traffic demand of the lane. The specific method is to obtain the data of vehicle passing and queue length in the past several signal cycles, and obtain the start and end time of the green light release of each lane in each signal cycle according to the matching relationship between the phase, flow direction and lane. The system intercepts the vehicle passing records according to these times and obtains the number of vehicles passing the lane during the green light time. At the same time, the system also obtains the queue length data within a few seconds before and after the end of the green light, and takes the maximum value as the queue length at the end of the green light of the lane in cycle C. The system combines these data to calculate the traffic demand of each lane in cycle C, and averages the traffic demand of m signal cycles to obtain the average traffic demand of the lane.

[0091] After calculating the average traffic demand of the lane, the system needs to estimate the green light duration of three phases for the opposite entrance with imbalance alarm, namely phase P1 for the straight flow at the opposite entrance, phase P2 for the left straight flow at the entrance with heavy traffic, and phase P3 for the left turn flow at the opposite entrance. For phases P1 and P3, the system estimates the green light duration based on traffic demand, and compares it with the maximum and minimum green light durations of the phase to determine the final recommended green light duration. Specifically, if the estimated green light duration is greater than or equal to the minimum green light duration of the phase and less than or equal to the maximum green light duration, the recommended green light duration is the estimated value; if it is less than the minimum green light duration, the minimum green light duration is recommended; if it is greater than the maximum green light duration, the maximum green light duration is recommended.

[0092] In summary, the advantages of the three-dimensional map generation method and system based on video recognition designed by the present invention are: 1. Automatic matching: Using advanced image recognition technology, the real-time images captured by the camera are matched with the real-time images of each point on the 3D map to achieve automatic switching to a viewing angle with high similarity, completely avoiding the process of manual data entry, thereby greatly reducing workload and improving efficiency; 2. Improve accuracy: Use computer vision algorithms to perform image similarity matching, reduce human errors, and improve the accuracy of the correspondence between camera images and 3D map point perspectives; 3. Simplified maintenance: The system can automatically match and update the correspondence between the camera image and the three-dimensional map perspective in real time, which greatly simplifies system maintenance and improves the flexibility and adaptability of the system.

[0093] The purpose of the present invention is to solve the problems of low efficiency, easy errors and difficult maintenance caused by manual input in the prior art through automated image recognition and matching technology, and to realize a more efficient, accurate and easy-to-maintain three-dimensional map perspective automatic switching system.

[0094] The present invention is not limited to the above-mentioned embodiments. No matter any changes are made in the shape or material composition, any structural design provided by the present invention is a variation of the present invention and should be considered to be within the protection scope of the present invention.

Claims

1. A three-dimensional map generation method based on video recognition, characterized in that: The following steps are involved: S1: Initialize each module, capture the image in real time and perform preliminary processing on the image; S2: 3D rendering of the map screen at a specific viewing angle according to the command; S3: pre-processing the 3D map rendering image and the real-time image after preliminary processing; S4: using an image similarity algorithm to calculate the similarity of the two processed images; S5: Generate a qualified three-dimensional map rendering image and store the update in a database.

2. A three-dimensional map generation method based on video recognition according to claim 1, characterized in that: The step S3 includes the following steps: S3.1: Acquire the original image to be processed, wherein the original image includes a three-dimensional map rendering image and a real-time image after preliminary processing; S3.2: resize the image to the desired size and convert the color image to grayscale image; S3.3: Use edge detection to extract image edge features; S3.4: Extract image feature points by establishing Gaussian pyramid and difference Gaussian pyramid; S3.5: Save or display the processed image and output the detected edge and feature point information.

3. The method for generating a three-dimensional map based on video recognition according to claim 2, characterized in that: The step S3.3 includes the following steps: S3.31: Use Gaussian filter to smooth the image; S3.32: Calculate the gradient of the image in the x and y directions; S3.33: Perform non-maximum suppression on the gradient amplitude and retain the local maximum as a potential edge point; S3.34: Use high and low thresholds to classify the gradient amplitude and determine strong and weak edges; S3.35: Connect strong edges and weak edges through hysteresis thresholding to form a complete edge image.

4. A three-dimensional map generation method based on video recognition according to claim 1, 2 or 3, characterized in that: The step S4 includes the following steps: S4.1: receiving two pre-processed images to be compared; S4.2: Extract features from the image, perform feature matching on the extracted features, and apply ratio test to select high-quality matches; S4.3: Perform geometric verification on the matching results to remove incorrect matches; S4.4: Calculate the similarity and determine whether they are similar according to the set threshold. If so, proceed to step S5; if not, proceed to step S4.5; S4.5: The control module instructs the three-dimensional map module to adjust the viewing angle according to a predetermined step length, and returns to step S2.

5. A three-dimensional map generation method based on video recognition according to claim 2 or 3, characterized in that: In step S3.2, adjusting the image size to the desired size includes adjusting the image size to the desired size using bilinear interpolation or nearest neighbor interpolation.

6. A three-dimensional map generation method based on video recognition according to claim 2 or 3, characterized in that: The step S3.4 includes the following steps: S3.41: Generate Gaussian pyramid and difference Gaussian pyramid; S3.42: Find potential feature points in scale space and accurately locate the feature points by interpolation; S3.43: Assign a main direction to each keypoint and generate a description vector based on the area around the keypoint.

7. The method for generating a three-dimensional map based on video recognition according to claim 1, characterized in that: In the step S1, the initialization of each module includes a control module loading initial parameters, and the initial parameters include a similarity threshold and a viewing angle adjustment step size; the real-time capture of the picture and the preliminary processing of the picture include a camera module continuously capturing the real-time picture, transmitting the digital signal to the image processing unit through the image acquisition card, and the image processing unit performing preliminary processing on the received picture, and the preliminary processing includes denoising and adjusting the contrast; in the step S2, the three-dimensional rendering of the map picture of a specific viewing angle according to the instruction includes the three-dimensional map module rendering the map picture of a specific viewing angle according to the instruction of the control module, and transmitting it to the image processing unit after the rendering is completed.

8. The method for generating a three-dimensional map based on video recognition according to claim 1, characterized in that: The step S5 includes: generating a qualified three-dimensional map rendering image, the control module sending the correspondence between the real-time picture and the three-dimensional map perspective to the database module for storage, the database module updating the stored correspondence, and returning to the step of capturing the picture in real time and performing preliminary processing on the picture in step S1; the relevant data includes a timestamp, a camera ID, and three-dimensional map perspective parameters.

9. A three-dimensional map generation system based on video recognition, using a three-dimensional map generation method based on video recognition according to any one of claims 1 to 8, characterized in that: include: A three-dimensional map module and a control module, wherein the three-dimensional map module is electrically connected to an image processing unit, wherein the image processing unit is communicatively connected to a camera module, and wherein the control module is electrically connected to the three-dimensional map module, a database module and an image recognition module, wherein the image recognition module is connected to the image processing unit.

10. A three-dimensional map generation system based on video recognition according to claim 9, characterized in that: The camera module captures video images in real time, the image module preprocesses the image and calculates similarity, the three-dimensional map module performs multi-perspective three-dimensional rendering of the map, the control module switches the perspective and calculates similarity, and the database module saves the similarity and the corresponding relationship after successful comparison.

Images