A three-dimensional modeling method and system based on cooperation of a mobile phone and a drone in shooting

Abstract

The application provides a three-dimensional modeling method and system based on cooperation of a mobile phone and a UAV in shooting, comprising a data acquisition module, a data transmission module, a data processing center and a model display and analysis module; the data acquisition module adopts an acquisition strategy configuration logic based on a traffic accident type and combined with an accident scene scale, and is used for determining a combination mode of acquisition equipment under different traffic accident scenes; the data acquisition module comprises a mobile phone shooting unit used for close-range data acquisition of a small-range traffic accident scene and a UAV shooting unit used for data acquisition of a large-range traffic accident. Through the cooperative acquisition mode of the mobile phone and the UAV and combined with a three-dimensional modeling processing flow, macro scene information and micro detail information can be simultaneously acquired at an accident scene, the acquisition and modeling of key elements of the accident scene can be completed in a short time, and the application is favorable for shortening the occupation time of the accident scene and improving traffic relief efficiency.

Description

Technical Field

[0001] This invention relates to the field of surveying and mapping and 3D modeling fusion technology in traffic accident handling, and particularly to a 3D modeling method and system based on collaborative shooting by mobile phone and drone. Background Technology

[0002] Road traffic accident scene investigation is a crucial step in accident handling and liability determination. It typically requires the rapid and accurate collection and recording of vehicle positions, road structure, collision marks, and related evidence at the accident scene, while ensuring traffic safety and efficiency. Traditional accident investigation methods rely heavily on manual photography, measurement, and drawing, which are cumbersome, time-consuming, and the consistency of results is greatly affected by the experience of the personnel, making it difficult to meet the practical needs of "rapid investigation and rapid removal." With the development of image acquisition and 3D modeling technologies, existing technologies are gradually incorporating various devices such as drones and smartphones to collect data from traffic accident scenes and perform 3D reconstruction based on methods such as photogrammetry. Among these, drones can acquire the overall layout and road structure information of the accident scene from an aerial perspective, while smartphones can be used to collect close-up details such as damaged vehicle parts, brake marks, and debris, thus forming a modeling basis for multi-source data fusion. Such technologies have improved the completeness and intuitiveness of accident scene information acquisition to a certain extent and optimized the shortcomings of traditional investigation methods. However, existing multi-device data acquisition and modeling solutions still have shortcomings in practical applications: On the one hand, different types and scales of traffic accidents have different requirements for data acquisition focus and methods. Existing technologies generally lack clear data acquisition strategies for accident types and scene scales, which can easily lead to insufficient coverage of key areas. On the other hand, the communication environment at accident scenes is complex. Relying solely on cloud or online transmission methods makes it difficult to guarantee the timeliness of data processing under weak or no network conditions. At the same time, existing modeling processes often lack real-time verification and supplementary acquisition mechanisms for the completeness of acquired data coverage. Once data loss is detected, it often requires a second round of reshoots, which affects the efficiency of accident handling. To address this, a 3D modeling method and system based on collaborative shooting by mobile phones and drones is proposed. Summary of the Invention

[0003] In view of this, the present invention provides a 3D modeling method and system based on collaborative shooting by mobile phone and drone, so as to solve or alleviate the technical problems existing in the prior art, and at least provide a beneficial option.

[0004] The technical solution of the present invention is achieved in the following way: a 3D modeling system based on collaborative shooting by mobile phone and drone, including a data acquisition module, a data transmission module, a data processing center and a model display and analysis module; The data acquisition module employs a data acquisition strategy configuration logic based on traffic accident type and accident scenario scale to determine the combination of data acquisition devices under different traffic accident scenarios. The data acquisition module includes: The mobile phone camera unit is used to collect close-up data of small-scale traffic accident scenes to obtain detailed feature information of vehicle damage, brake marks, and scattered objects. The drone photography unit is equipped with a GPS + Beidou dual-mode positioning module and RTK real-time differential technology to conduct aerial photography of large-area traffic accident scenes in order to obtain information on the range of lanes involved in the accident, the spatial distribution of vehicles, and macroscopic scene information of traffic signs and markings. The data transmission module is used to transmit the data collected by the mobile phone shooting unit and the drone shooting unit to the data processing center; The data processing center includes mobile computing devices and cloud servers. The data processing center has a built-in data processing module for format unification, spatial coordinate alignment and image distortion correction of data from mobile phone shooting units and drone shooting units. On this basis, it completes the fusion modeling of macro data and micro data, and verifies the coverage integrity of the collected data and outputs supplementary collection instruction information. The model display and analysis module is used to visualize the generated 3D model and supports measurement and annotation operations on the 3D model to generate accident scene result data that conforms to the road traffic accident scene drawing specifications.

[0005] More preferably, in the data transmission module, the data collected by the mobile phone shooting unit is imported into the mobile computing device via a removable storage medium or uploaded to the cloud server via a communication network, and the data collected by the drone shooting unit is imported into the mobile computing device via a removable storage medium.

[0006] More preferably, the mobile computing device is deployed at the scene of a traffic accident to complete data reception, processing, and data coverage integrity verification without relying on an external communication network.

[0007] More preferably, the data processing module is used to uniformly convert data from the mobile phone shooting unit and the drone shooting unit into point cloud data and establish a unified spatial coordinate system.

[0008] Furthermore, the model display and analysis module further includes a backend management unit for encrypted storage, access control, and data traceability management of traffic accident data.

[0009] More preferably, the drone shooting unit is suitable for large-scale traffic accident scenarios on highways, national roads, or large intersections, while the mobile phone shooting unit is suitable for small-scale traffic accident scenarios on urban roads or expressways.

[0010] This invention also provides a 3D modeling method based on collaborative shooting by a mobile phone and a drone, comprising the following steps: S1. Accident scene data collection: Based on the type of traffic accident and the scale of the accident scene, select a mobile phone shooting unit, a drone shooting unit, or a combination of both to collect image data of the accident scene. S2. Data transmission and processing: The acquired image data is transmitted to a mobile computing device or cloud server. The image data is processed by format unification, image distortion correction, spatial coordinate alignment, and fusion of macro and micro data. The three-dimensional model is reconstructed based on photogrammetry. At the same time, the coverage integrity of the acquired data is verified. S3. Model Display and Analysis: Display the generated 3D model and perform measurement and annotation operations on the 3D model to generate traffic accident scene analysis results data.

[0011] In a further preferred embodiment, in step S1, a small-scale traffic accident scene is captured by a mobile phone camera unit for surround shooting, while a large-scale traffic accident scene is captured by a drone camera unit for aerial shooting. The mobile phone camera unit also collects additional key details as needed.

[0012] In a further preferred embodiment, in step S2, when it is detected that the collected data does not cover the preset area of ​​the accident site, a supplementary collection prompt message is output to guide the execution of regional supplementary collection.

[0013] More preferably, the data processing in step S2 is performed by a mobile computing device or a cloud server, depending on the on-site communication conditions.

[0014] The embodiments of the present invention have the following advantages due to the adoption of the above technical solutions: I. This invention uses a collaborative data collection method combining mobile phones and drones, along with a 3D modeling process, to simultaneously acquire macroscopic scene information and microscopic detail information at the accident site. Compared to methods that rely solely on manual measurement or single-device data collection, this invention can complete the collection and modeling of key elements at the accident site in a shorter time, which helps to shorten the time spent at the accident site and improve traffic management efficiency. Second, this invention provides measurement and annotation functions based on a three-dimensional model. Relevant personnel can automatically obtain measurement results such as distance and positional relationships by selecting measurement points in the model, avoiding the problems of complex operation and large error caused by manual measurement by means of measuring tape in traditional on-site investigation; at the same time, the system can generate road traffic accident scene diagrams that conform to the GA / T 49-2019 standard, reducing manual drawing steps and improving drawing efficiency and consistency of results; Third, this invention manages accident data in a unified manner through a back-end management module, enabling encrypted storage, access control, and multi-dimensional querying of data. This creates a complete data management chain for accident scene data in the stages of collection, processing, storage, and retrieval, improving the standardization and traceability of accident evidence and facilitating subsequent accident handling and review. Fourth, this invention adopts a modular system structure and a process-oriented operation method, integrating an automated processing engine and a shooting guidance mechanism. It standardizes the core links such as accident scene collection and modeling process, enabling non-professional modelers to complete accident scene investigation and result generation according to the process. This reduces the reliance on professional skills and fully meets the practical needs of front-line traffic accident handling, enhancing the applicability of the technical solution in the scenario.

[0015] The above overview is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of the invention will become readily apparent from the accompanying drawings and the following detailed description. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a system architecture diagram of the present invention; Figure 2 This is a diagram illustrating the method steps of the present invention; Figure 3 This is an example of a mobile phone data acquisition model for the present invention; Figure 4 This is an example of a mobile phone data acquisition model for the present invention; Figure 5 This is an example of a drone data acquisition model for the present invention; Figure 6 This is an example of a drone data acquisition model for the present invention. Detailed Implementation

[0018] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0019] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0020] like Figure 1 As shown, this embodiment of the invention provides a 3D modeling system based on collaborative shooting by a mobile phone and a drone.

[0021] This invention is mainly applied to traffic accident scene investigation scenarios. Through a dual-dimensional data collection logic of "accident type as the main factor + scene size as the auxiliary factor", combined with a dual-mode processing mode of mobile computing devices and cloud servers, it can achieve rapid investigation of accident scenes, high-precision 3D modeling, and output of results that meet standards.

[0022] In this embodiment, the system architecture of the present invention is divided into four modules: data acquisition module, data transmission module, data processing center, and model display and analysis module.

[0023] The data acquisition module includes a mobile phone shooting unit and a drone shooting unit, and adopts a two-dimensional acquisition logic of "accident type as the main factor + scene size as the auxiliary factor" to clarify the acquisition strategy and equipment adaptation relationship for different accident types.

[0024] The mobile phone camera unit is used for capturing detailed images of small-scale accident scenes. Law enforcement officers can use the mobile phone camera to take panoramic photos of the accident scene, capturing details such as damaged vehicle parts, skid marks, and debris. To improve image stability and clarity, mobile phone photography is best performed with a tripod or gimbal for stabilization, avoiding loss of detail due to hand shake.

[0025] The drone photography unit is used for macroscopic information collection in large-scale accident scenarios. The drone can be equipped with a high-definition camera module, a GPS+BeiDou dual-mode positioning module, and RTK real-time differential technology. It can quickly reach the airspace above the accident site and acquire macroscopic information such as the range of lanes involved in the accident, the position of vehicles, and surrounding traffic signs and markings through top-down and surround shooting methods. This provides global geometric constraints and scene structure information for subsequent modeling.

[0026] With the above dual-unit configuration, this system can choose to use only the mobile phone shooting unit or use the drone shooting unit in addition to the mobile phone shooting unit, depending on the type of accident and the scale of the scene, so as to avoid the problem of missing information caused by single device collection.

[0027] The data transmission module is used to transmit data captured by mobile phones and drones to the data processing center. The transmission method can be selected according to the on-site communication conditions and operating habits, preferably including one or a combination of the following methods: Data captured by mobile phones can be exported to mobile computing devices via removable storage media such as USB flash drives; and / or uploaded to cloud servers in real time via mobile networks; data captured by drones can be exported to mobile computing devices via removable storage media such as USB flash drives, ensuring that large-capacity image data can be reliably transmitted under weak network or no network conditions.

[0028] By combining two transmission paths—"removable storage media and communication networks"—the needs for real-time on-site processing can be met, as well as data backup and fault tolerance can be achieved, thereby improving data availability under conditions of rapid on-site investigation and withdrawal.

[0029] The data processing center consists of mobile computing devices and cloud servers. It can select the processing method according to the on-site network conditions, and all of them have the ability to verify the integrity of the collected data and provide guidance on supplementary data collection.

[0030] Mobile computing devices can be deployed directly at the accident site to receive and process data captured by mobile phones and / or drones without relying on external networks. During the processing, the mobile computing devices can instantly verify whether the data collection is complete and valid, and provide regional reshoot suggestions based on the current coverage, avoiding the need for a second return to the site for reshooting due to data loss after the site is evacuated.

[0031] The cloud server receives data uploaded from the site via the Internet, and performs in-depth processing using professional modeling software and high-performance computing power. It also performs real-time integrity verification and validity judgment on the collected data, balancing on-site processing efficiency with backend modeling accuracy.

[0032] The data processing center has a built-in data standardization processing unit, which preferably uses the self-developed M3D processing module to standardize the data collected by mobile phones and drones. The process involves converting multi-source image data into point cloud data; establishing a unified spatial coordinate system and aligning the coordinates; correcting distortion in UAV imagery; achieving seamless integration of macro and micro data; verifying data coverage integrity; and outputting supplementary data collection instructions.

[0033] Through the above processing, the data processing center can generate a three-dimensional model that meets the needs of accident investigation and mapping applications, and preferably achieves a high-precision restoration of the accident scene.

[0034] The model display and analysis module can display 3D models via computers or mobile devices, supporting operations such as measurement and annotation on the model. This facilitates measurement and analysis of accident scenes by traffic police, insurance adjusters, and other relevant personnel. The module can also generate road traffic accident scene diagrams and other output data conforming to the GA / T49-2019 standard, which can be used for evidence preservation and transfer in business processes such as accident liability determination, insurance loss assessment, and scene reconstruction.

[0035] like Figure 2 As shown, based on the above system, the present invention also provides a 3D modeling method based on collaborative shooting by a mobile phone and a drone, including the following steps: Step S1: Accident scene data collection After the accident, law enforcement officers determined the scope and complexity of the accident scene based on the principle of "accident type as the primary factor and scene size as the secondary factor," and selected the combination of data collection equipment accordingly. For minor collisions or other small-scale accidents, it is preferable to use a mobile phone camera unit to take panoramic photos to obtain detailed information; preferably, on-site data collection can be completed in a short time. For multi-vehicle accidents, large intersections, or large-scale accident scenes on highways, it is preferable to use drone shooting units to take aerial and surround shots to obtain macro information, while mobile phone shooting units can supplement the collection of key details.

[0036] Step S2, Data Transmission and Processing Data captured by mobile phones and drones is exported to mobile computing devices for processing via removable storage media, and / or data captured by mobile phones is uploaded to cloud servers in real time for processing; the mobile computing devices and cloud servers call the data standardization processing unit (preferably the M3D processing module) to perform format unification, distortion correction, coordinate alignment and fusion modeling of multi-source data, and verify the data coverage integrity; When a key area is detected to be not effectively covered or has insufficient clarity, the system outputs a supplementary shooting prompt to guide law enforcement personnel to carry out regional supplementary shooting and updates the model after the supplementary shooting data is imported / uploaded.

[0037] Step S3: Model Display and Analysis Relevant personnel can view the 3D model via mobile terminals or computers, and define benchmark points, baselines, and measurement points on the model. The system automatically calculates the corresponding distances, orientations, and other measurement results. At the same time, it outputs a road traffic accident scene diagram that conforms to the GA / T49-2019 standard, which serves as valid evidence for accident handling and is used for subsequent analysis and archiving.

[0038] Example 1: Implementation of Small-Scale Traffic Accident Scene Modeling (corresponding to) Figure 3 , Figure 4(Example of a mobile phone data acquisition model) (a) Implementation Scenarios This embodiment is applicable to small-scale traffic accident scenarios on urban roads and expressways, such as accidents involving two vehicles colliding and minor injuries. In such scenarios, the road occupancy area is small and the macro layout is relatively simple, but there are high requirements for the clear collection of detailed information such as collision marks, brake marks and debris. At the same time, it is necessary to complete the investigation as soon as possible to restore traffic flow.

[0039] (II) Implementation Process After arriving at the scene, traffic police will activate an accident scene photography application or follow the on-site investigation procedures to take photos around the vehicles involved in the accident. During the photography process, the focus should be on clearly capturing the collision points between the two vehicles, scratches on the paint, brake marks, and debris scattered around the vehicles. Using a tripod or gimbal is recommended to improve shooting stability and image clarity. After shooting, the data captured by the mobile phone can be uploaded to the cloud server via mobile network; when the network conditions are unstable or when rapid on-site processing is required, it can also be exported to a mobile computing device for local processing via USB flash drive or other means. The data processing center performs standardized processing and integrity verification on the data: if a key detail area is found to be under-covered or the clarity does not meet the modeling requirements, a supplementary shooting prompt is output to the mobile terminal to guide law enforcement personnel to take supplementary photos of the designated area. After the supplementary photos are completed and imported / uploaded, the system updates the 3D model and generates on-site map results that conform to the GA / T49-2019 standard for use in liability determination and damage assessment.

[0040] Example 2: Implementation of Large-Scale Traffic Accident Scene Modeling (corresponding to) Figure 5 , Figure 6 (Example of a drone data collection model) (a) Implementation Scenarios This embodiment is applicable to large-scale traffic accident scenarios on highways, national roads, and large intersections, such as multi-vehicle rear-end collisions. These scenarios involve a large number of vehicles and a large area of ​​road occupancy. It is necessary to collect macro information such as vehicle distribution, lane range, and road markings, as well as key vehicle collision details, and to complete the investigation quickly to alleviate traffic congestion.

[0041] (II) Implementation Process Traffic police activated the drone photography unit within the safe area of ​​the scene. With the support of GPS+BeiDou dual-mode positioning and RTK real-time differential technology, the drone flew over the accident scene to perform overhead and surround photography, collecting macroscopic information such as the range of lanes involved in the accident, the position of each vehicle, and the damage to surrounding traffic signs, markings, and road facilities. At the same time, other traffic police officers at the scene used mobile phone photography units to supplement the collection of key details (such as the collision points of severely damaged vehicles, brake marks, and the distribution of scattered objects). After the shooting is completed, the data collected by the drone and mobile phone is exported to the mobile computing device on site via USB flash drive or other means for standardization processing, coordinate unification and fusion modeling, and integrity verification is performed; if the edge of a lane, damaged section of facilities or marking area is found to be under-covered, a supplementary shooting prompt is output to guide the drone or mobile phone to take supplementary shots in the area, and the model is updated after the supplementary shooting data is imported. The final generated 3D model and site map can be used for traffic management decisions, accident cause analysis, liability determination, insurance loss assessment, and subsequent accident site reconstruction. Compared with traditional on-site measurement and manual drawing processes, this implementation method can reduce on-site operation steps and improve the standardization of results. The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in the present invention, and these should all be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A 3D modeling system based on collaborative shooting by mobile phone and drone, characterized in that, It includes a data acquisition module, a data transmission module, a data processing center, and a model display and analysis module; The data acquisition module employs a data acquisition strategy configuration logic based on traffic accident type and accident scenario scale to determine the combination of data acquisition devices under different traffic accident scenarios. The data acquisition module includes: The mobile phone camera unit is used to collect close-up data of small-scale traffic accident scenes to obtain detailed feature information of vehicle damage, brake marks, and scattered objects. The drone photography unit is equipped with a GPS + Beidou dual-mode positioning module and RTK real-time differential technology to conduct aerial photography of large-area traffic accident scenes in order to obtain information on the range of lanes involved in the accident, the spatial distribution of vehicles, and macroscopic scene information of traffic signs and markings. The data transmission module is used to transmit the data collected by the mobile phone shooting unit and the drone shooting unit to the data processing center; The data processing center includes mobile computing devices and cloud servers. The data processing center has a built-in data processing module for format unification, spatial coordinate alignment and image distortion correction of data from mobile phone shooting units and drone shooting units. On this basis, it completes the fusion modeling of macro data and micro data, and verifies the coverage integrity of the collected data and outputs supplementary collection instruction information. The model display and analysis module is used to visualize the generated 3D model and supports measurement and annotation operations on the 3D model to generate accident scene result data that conforms to the road traffic accident scene drawing specifications.

2. The 3D modeling system based on collaborative shooting by mobile phone and drone as described in claim 1, characterized in that, In the data transmission module, the data collected by the mobile phone shooting unit is imported into the mobile computing device via a removable storage medium or uploaded to the cloud server via a communication network, and the data collected by the drone shooting unit is imported into the mobile computing device via a removable storage medium.

3. The 3D modeling system based on collaborative shooting by mobile phone and drone as described in claim 1, characterized in that, The mobile computing device is deployed at the scene of a traffic accident to receive, process, and verify the integrity of collected data coverage without relying on an external communication network.

4. The 3D modeling system based on collaborative shooting by mobile phone and drone as described in claim 1, characterized in that, The data processing module is used to convert data from the mobile phone shooting unit and the drone shooting unit into point cloud data and establish a unified spatial coordinate system.

5. The 3D modeling system based on collaborative shooting by mobile phone and drone as described in claim 1, characterized in that, The model display and analysis module also includes a back-end management unit for encrypted storage, access control, and data traceability management of traffic accident data.

6. The 3D modeling system based on collaborative shooting by mobile phone and drone as described in claim 1, characterized in that, The drone shooting unit is suitable for large-scale traffic accident scenarios on highways, national roads, or large intersections, while the mobile phone shooting unit is suitable for small-scale traffic accident scenarios on urban roads or expressways.

7. A 3D modeling method based on collaborative shooting by mobile phone and drone, characterized in that, Includes the following steps: S1. Accident scene data collection: Based on the type of traffic accident and the scale of the accident scene, select a mobile phone shooting unit, a drone shooting unit, or a combination of both to collect image data of the accident scene. S2. Data transmission and processing: The acquired image data is transmitted to a mobile computing device or cloud server. The image data is processed by format unification, image distortion correction, spatial coordinate alignment, and fusion of macro and micro data. The three-dimensional model is reconstructed based on photogrammetry. At the same time, the coverage integrity of the acquired data is verified. S3. Model Display and Analysis: Display the generated 3D model and perform measurement and annotation operations on the 3D model to generate traffic accident scene analysis results data.

8. The 3D modeling method based on collaborative shooting by mobile phone and drone as described in claim 7, characterized in that, In step S1, small-scale traffic accident scenes are captured by a mobile phone camera unit, while large-scale traffic accident scenes are captured by a drone camera unit. The mobile phone camera unit also collects additional data on key details as needed.

9. The 3D modeling method based on collaborative shooting by mobile phone and drone as described in claim 7, characterized in that, In step S2, when it is detected that the collected data does not cover the preset area of ​​the accident site, a supplementary collection prompt message is output to guide the execution of regional supplementary photography.

10. The 3D modeling method based on collaborative shooting by mobile phone and drone as described in claim 7, characterized in that, The data processing in step S2 is performed by either a mobile computing device or a cloud server, depending on the on-site communication conditions.

Images