A real-time digital parking lot construction method based on video 3D reconstruction
By filtering the duplicate areas and obtaining depth information, the alignment process is simplified and the three-dimensional model is generated using neural networks, which solves the problem of low repetitive modeling and alignment efficiency in existing real-time digital car parks, and achieves efficient and accurate real-time digital car park construction.
Patent Information
- Application Number
- CN202411169882.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-08-26
AI Technical Summary
The existing real-time digital car park construction methods have problems such as duplicate modeling, inability to obtain depth information and monitoring video targets and three-dimensional map alignment efficiency, resulting in display errors and delays.
By filtering information on monitoring overlapping areas, using video information from the current moment and past time periods to obtain the depth information of the target, and simplifying the alignment method of monitoring video targets with the three-dimensional map, a three-dimensional model is generated using neural networks and Transformer networks.
Repeated modeling is avoided, the accuracy and alignment efficiency of the three-dimensional model are improved, and the real-time requirements of real-time digital car parks are met.
Smart Images

Figure CN118674875B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of three-dimensional modeling, and in particular to a real-time digital parking lot construction method based on video three-dimensional reconstruction. Background Art
[0002] Real-time digital parking lot is an intelligent parking lot management technology that combines advanced technologies such as 3D GIS, video surveillance and artificial intelligence. It unifies scattered video data into a three-dimensional model through real-time monitoring and data analysis, and realizes unified management and optimization of vehicles and facilities inside and outside the parking lot. Compared with traditional parking lot management, real-time digital parking lot can make full use of monitoring data, conduct real-time data analysis and anomaly detection, improve safety, and unify scattered monitoring images into the same three-dimensional scene, which can comprehensively and efficiently monitor the parking lot and its surrounding environment.
[0003] The existing real-time digital parking lot construction method has the following shortcomings: (1) some monitoring areas are modeled repeatedly, resulting in display errors in the final digital parking lot; (2) excessive reliance on the current video frame information, unable to obtain the depth information of the target, resulting in poor reconstructed 3D model effect; (3) the alignment method of the 3D model and 3D map of the monitoring video target is cumbersome and inefficient, resulting in display delays in the real-time digital parking lot. Therefore, an efficient and accurate real-time digital parking lot construction method is needed. Summary of the invention
[0004] In view of the above-mentioned problems, the present invention proposes a real-time digital parking lot construction method based on video three-dimensional reconstruction, which avoids repeated modeling of the same target by filtering the information of the overlapping monitoring areas; at the same time, the depth information of the target is obtained by using the video information of the current moment and the past time period, thereby improving the accuracy of the three-dimensional model; finally, by simplifying the three-dimensional model of the monitoring video target and the three-dimensional map alignment method, the real-time performance of the digital parking lot is guaranteed.
[0005] To achieve the above object, the present invention is implemented through the following technical solutions:
[0006] A real-time digital parking lot construction method based on video three-dimensional reconstruction includes the following steps:
[0007] S1. Build a three-dimensional map by first obtaining the latitude and longitude coordinates of the parking lot center and the length and width of the parking lot and its surroundings, and then use them as input to the open source 3D GIS software gVSIG to obtain a digital model of the parking lot and its surroundings;
[0008] S2, aligning the monitoring coverage area with the three-dimensional map, transforming the parking lot digital model into a new parking lot digital model according to the monitoring tilt angle, and directly aligning the monitoring coverage area with the three-dimensional map by scaling the new parking lot digital model;
[0009] S3. Obtain video data, number all surveillances, and obtain the current time and previous time of each surveillance. k Video clips of the moment;
[0010] S4, video target analysis, determining the target in the video and the target's attribute information through a target detection algorithm, wherein the attribute information includes the two-dimensional coordinates of the key points of the target;
[0011] S5, generating a three-dimensional model of the target, first using a neural network to convert the two-dimensional coordinates of the target key points into three-dimensional key points, and then using a neural network to generate a three-dimensional model based on the three-dimensional key points, wherein the three-dimensional model is composed of multiple triangular meshes, and the three-dimensional mesh vertex coordinates are scaled according to the proportional relationship between the monitoring coverage area and the three-dimensional map to obtain a three-dimensional model of the target in the digital parking lot;
[0012] S6, synthesizing the digital parking lot, merging the three-dimensional models of all targets included in the monitoring at a certain moment with the three-dimensional map to form the digital parking lot at the current moment.
[0013] As a further technical solution of the present invention, the specific process of step S1 is:
[0014] S11. Obtain the longitude and latitude coordinates of the center of the target parking lot ( x 车场 , y 车场 ); At the same time, obtain the length and width of the parking lot and its surrounding area ( h 车场 , w 车场 );
[0015] S12. The open source 3D GIS software gVSIG is represented as , the three-dimensional map created for: , 3D map is a set of points, that is, {( x 0 , y 0 , z 0 , c , σ )},in x 0 for point x Coordinate values, y 0 for point y Coordinate values, z 0 for point z Coordinate values,c is the color of the point, σ is the transmittance of the point.
[0016] As a further technical solution of the present invention, the new parking lot three-dimensional map obtained in step S2 is: ,in , , , x 0 The origin of the point x Coordinate values, y 0 The origin of the point y Coordinate values, z 0 The origin of the point z Coordinate values, x After alignment x Coordinate values, y After alignment y Coordinate values, z After alignment z Coordinate values, θ To monitor the tilt angle θ .
[0017] As a further technical solution of the present invention, the specific process of step S4 is:
[0018] S41, detecting two-dimensional key points, for any video frame of any monitoring, the target detection process is expressed as: ,in represents the two-dimensional key point detection algorithm, Indicates n Monitor at all times t The video frame data, Indicates n Monitor at all times t The target set contained in the video frame is ,in , Indicates n Monitor at all times t The video frame contains the s goals, S Indicates n Monitor at all times t The total number of objects contained in the video frames, Indicates s The first target j The key two-dimensional coordinates, J Indicates s The number of key points of each target;
[0019] S42. Filter duplicate targets. For the same targets existing in adjacent monitors, select the monitor with the smaller number. For the i-th monitor and the j-th monitor, where i < j, the targets they contain at time t are and . By traversing each target among them and calculating the similarity, if the similarity exceeds the threshold α , then remove this target from the j-th monitor. The similarity calculation function is: , where represents the s -th target of the i-th monitor, represents the s -th target of the j-th monitor, and represents the similarity.
[0020] As a further technical solution of the present invention, the specific process of step S5 is as follows:
[0021] S51. First, based on the two-dimensional key points of the target, use a neural network to map them into three-dimensional key points. For a single target, the generation process of its three-dimensional key points is expressed as: , where represents the neural network, represents the s -th three-dimensional key point of the target, and the set of three-dimensional key points of the s -th target is specifically expressed as , where , respectively represent the x coordinate value, y coordinate value, z coordinate value, color value, and transmittance of the j-th key point;
[0022] S52. Based on the three-dimensional key point coordinates, use a Transformer neural network to generate a three-dimensional model. The three-dimensional model generation process is expressed as: , where represents the Transformer neural network, represents the three-dimensional model, and represents t the target at time
[0023] As a further technical solution of the present invention, the specific process of step S6 is expressed as: , where represents t the digital parking lot at time represents the three-dimensional map, represents the scaling ratio, represents the three-dimensional model of the t -th target of the n-th monitor at time s .
[0024] Compared with the prior art, the present invention has the following beneficial effects:
[0025] Compared with the existing real-time digital parking lot, the method proposed in the present invention can filter targets in repeated areas, thereby ensuring the uniqueness of the targets, avoiding repeated modeling, and improving modeling efficiency.
[0026] Existing methods usually perform 3D modeling based only on the current video frame. However, a single frame of video lacks depth information of the target, and depth information is an important factor in ensuring the accuracy of the 3D model. Therefore, the accuracy of the 3D model reconstructed by the existing methods is low. The modeling method proposed in the present invention not only uses the current video frame, but also uses the previous video frame. k The depth information of the target can be easily obtained from the video frame at each moment and the video clip, so the present invention improves the accuracy of the three-dimensional model.
[0027] The present invention achieves a simplified method of aligning the monitoring screen and the three-dimensional map by scaling by adjusting the three-dimensional map coordinates, avoiding tedious calculation processes, greatly improving the alignment efficiency, and meeting the real-time requirements of real-time data parking lots. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their description are used to explain the present disclosure and do not constitute improper limitations on the present invention.
[0029] Figure 1 It is a flow chart of a real-time digital parking lot construction method based on video three-dimensional reconstruction provided by the present invention.
[0030] Figure 2 It is a schematic diagram of the process of establishing a three-dimensional map of a parking lot and its surroundings provided by the present invention.
[0031] Figure 3 It is a schematic diagram of video data acquisition provided by the present invention.
[0032] Figure 4 It is a schematic diagram of performing video data analysis based on a target detection algorithm provided by the present invention.
[0033] Figure 5 It is a schematic diagram of a 3D modeling method based on video 3D reconstruction provided by the present invention. DETAILED DESCRIPTION
[0034] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.
[0035] A real-time digital parking lot construction method based on video 3D reconstruction according to an example of the present invention is described below with reference to the accompanying drawings.
[0036] Example:
[0037] like Figure 1 As shown, this embodiment provides a real-time digital parking lot construction method based on video 3D reconstruction, comprising the following steps:
[0038] S1. Create a three-dimensional map:
[0039] The real-time digital parking lot is efficient and safe. The first step to establish a real-time digital parking lot is to create a three-dimensional map of the parking lot and its surroundings. The three-dimensional map is used as the basis of the digital parking lot. Figure 2 As shown, the specific process is:
[0040] S11. Obtain the longitude and latitude coordinates of the center of the target parking lot ( x 车场 , y 车场 ); At the same time, obtain the length and width of the parking lot and its surrounding area ( h 车场 , w 车场 );
[0041] S12. The open source 3D GIS software gVSIG is represented as , the three-dimensional map created for: , 3D map is a set of points, that is, {( x 0 , y 0 , z 0 , c , σ )},in x 0 for point x Coordinate values, y 0 for point y Coordinate values, z 0 for point z Coordinate values, c is the color of the point, σ is the transmittance of the point.
[0042] S2. Align monitoring coverage area and 3D map:
[0043] In order to solve the problem of complex calculation in the existing method in aligning the monitoring coverage area and the three-dimensional map, this embodiment establishes a three-dimensional map based on the monitoring tilt angle, transforms the parking lot digital model into a new parking lot digital model according to the monitoring tilt angle, and based on the new parking lot digital model, the monitoring coverage area is directly aligned with the three-dimensional map by scaling; the new parking lot three-dimensional map is: ,in , , , x 0 The origin of the point x Coordinate values, y 0 The origin of the point y Coordinate values, z 0 The origin of the point z Coordinate values, x After alignment x Coordinate values, y After alignment y Coordinate values, z After alignment z Coordinate values, θ To monitor the tilt angle θ .
[0044] S3. Get video data:
[0045] like Figure 3 As shown, all monitoring is defined as ,in Indicates n Monitoring, N Indicates the total number of monitoring. n Monitor and get its current time T As well as before k Video clips of the moment ,in Indicates n Monitor at all times t In the specific operation process, except for the current moment T The video frames need to be acquired in real time. k The video clip at each moment can be obtained from the cache database.
[0046] S4. Video target analysis:
[0047] The target in the video and its attribute information are determined by the target detection algorithm, where the attribute information includes the two-dimensional coordinates of the key points of the target, such as Figure 4 As shown, the specific process is:
[0048] S41. Detect 2D key points. For any video frame of any monitored object, the object detection process is expressed as: , where represents the 2D key point detection algorithm, represents the n th monitored object at time t video frame data, represents the n th monitored object at time t the set of objects included in the video frame, and the obtained set of objects is specifically , where , represents the n th monitored object at time t the s th object included in the video frame, S represents the n th monitored object at time t the total number of objects included in the video frame, represents the s th object's j th key 2D coordinates, J represents the s th object's number of key points;
[0049] S42. Filter duplicate objects. For the same objects existing in adjacent monitored objects, select the monitored object with a smaller number. For the ith and jth monitored objects, where α i < j the objects they respectively include at time t s are and s respectively. By traversing each object and calculating the similarity, if the similarity exceeds the threshold
[0050]
[0051] as Figure 5 shown, the specific process is as follows:
[0052] S51. First, based on the 2D key points of the object, use a neural network to map them to 3D key points. For a single object, the 3D key point generation process is expressed as: , where represents the neural network, represents the sThe three-dimensional key points of the target, s The three-dimensional key point set of a target is specifically expressed as ,in , Respectively represent the x-coordinate value, y-coordinate value, z-coordinate value, color value and transmittance of the j-th key point;
[0053] S52. Based on the coordinates of the three-dimensional key points, a three-dimensional model is generated using a Transformer neural network. The three-dimensional model generation process is expressed as follows: ,in represents the Transformer neural network, Represents a three-dimensional model, express t The goal of the moment.
[0054] S6, Synthetic Digital Parking Lot:
[0055] For a single target, the 3D model is first scaled according to the scaling ratio of the monitoring coverage area and the virtual scene, and then superimposed with the 3D map. The specific operations are as follows: ,in, express t The digital parking lot of the moment, Represents a three-dimensional map, Indicates the scaling ratio. Indicates that the nth monitoring t Moment s A three-dimensional model of a target.
[0056] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
[0057] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.
Claims
1. A real-time digital parking lot construction method based on video 3D reconstruction, characterized in that: The following steps are involved: S1. Build a three-dimensional map by first obtaining the latitude and longitude coordinates of the parking lot center and the length and width of the parking lot and its surroundings, and then use them as input to the open source 3D GIS software gVSIG to obtain a digital model of the parking lot and its surroundings; S2, aligning the monitoring coverage area with the three-dimensional map, transforming the parking lot digital model into a new parking lot digital model according to the monitoring tilt angle, and directly aligning the monitoring coverage area with the three-dimensional map by scaling the new parking lot digital model; S3. Obtain video data, number all surveillances, and obtain the current time and previous time of each surveillance. k Video clips of the moment; S4, video target analysis, determining the target in the video and the target's attribute information through a target detection algorithm, wherein the attribute information includes the two-dimensional coordinates of the key points of the target; S5, generating a three-dimensional model of the target, first using a neural network to convert the two-dimensional coordinates of the target key points into three-dimensional key points, and then using a Transformer neural network to generate a three-dimensional model based on the three-dimensional key points, wherein the three-dimensional model is composed of multiple triangular meshes, and the three-dimensional mesh vertex coordinates are scaled according to the proportional relationship between the monitoring coverage area and the three-dimensional map to obtain a three-dimensional model of the target in the digital parking lot; S6, synthesizing the digital parking lot, merging the three-dimensional models of all targets included in the monitoring at a certain moment with the three-dimensional map to form the digital parking lot at the current moment.
2. The real-time digital parking lot construction method based on video 3D reconstruction according to claim 1 is characterized in that: The specific process of step S1 is: S11. Obtain the longitude and latitude coordinates of the center of the target parking lot ( x 车场 , y 车场 ); At the same time, obtain the length and width of the parking lot and its surrounding area ( h 车场 , w 车场 ); S12. The open source 3D GIS software gVSIG is represented as Based on the information obtained from S11, the open source software gVSIG was used to build a 3D map. for: , 3D map is a set of points, that is, {( x 0 , y 0 , z 0 , c , σ )},in x 0 for point x Coordinate values, y 0 for point y Coordinate values, z 0 for point z Coordinate values, c is the color of the point, σ is the transmittance of the point.
3. The real-time digital parking lot construction method based on video 3D reconstruction according to claim 2 is characterized in that: The new parking lot three-dimensional map obtained in step S2 is: ,in , , , x 0 The origin of the point x Coordinate values, y 0 The origin of the point y Coordinate values, z 0 The origin of the point z Coordinate values, x After alignment x Coordinate values, y After alignment y Coordinate values, z After alignment z Coordinate values, θ To monitor the tilt angle θ .
4. The real-time digital parking lot construction method based on video 3D reconstruction according to claim 3 is characterized in that: The specific process of step S4 is: S41, detecting two-dimensional key points, for any video frame of any monitoring, the target detection process is expressed as: ,in represents the two-dimensional key point detection algorithm, Indicates n Monitor at all times t The video frame data, Indicates n Monitor at all times t The target set contained in the video frame is ,in , Indicates n Monitor at all times t The video frame contains the s goals, S Indicates n Monitor at all times t The total number of objects contained in the video frames, Indicates s The first target j The key two-dimensional coordinates, J Indicates s The number of key points of each target; S42. Filter duplicate targets. For the same targets existing in adjacent monitors, select the monitor with a smaller number. For the $i$-th monitor and the $j$-th monitor, where $i < j$, the targets they contain at time $t$ are and . By traversing each target therein and calculating the similarity, if the similarity exceeds the threshold α , then remove this target from the $j$-th monitor. The similarity calculation function is: , where represents the s -th target of the $i$-th monitor, represents the s -th target of the $j$-th monitor, represents the similarity.
5. The real-time digital parking lot construction method based on video 3D reconstruction according to claim 4 is characterized in that: The specific process of step S5 is: S51, based on the two-dimensional key points of the target, use a neural network to map them into three-dimensional key points. For a single target, the three-dimensional key point generation process is expressed as: ,in represents a neural network, Indicates s The three-dimensional key points of the target, s The three-dimensional key point set of a target is specifically expressed as ,in , Respectively represent the x-coordinate value, y-coordinate value, z-coordinate value, color value and transmittance of the j-th key point; S52. Based on the coordinates of the three-dimensional key points, a three-dimensional model is generated using a Transformer neural network. The three-dimensional model generation process is expressed as follows: ,in represents the Transformer neural network, Represents a three-dimensional model, express t The goal of the moment.
6. The real-time digital parking lot construction method based on video 3D reconstruction according to claim 5 is characterized in that: The specific process of step S6 is expressed as follows: , in, express t The digital parking lot of the moment, Represents a three-dimensional map, Indicates the zoom ratio. Indicates that the nth monitoring t Moment s A three-dimensional model of a target.
