Vehicle appearance panorama acquisition and display method and system

Abstract

The invention discloses a vehicle appearance panorama acquisition and display method and system, and the method comprises the steps: obtaining vehicle image data and attitude / angle related sensor data through a mobile terminal; vehicle type information is determined based on the image data, and a multi-view-angle collection template set containing view angle templates and view angle information of the view angle templates is determined according to the vehicle type information; displaying a target view angle template in a view-finding preview interface in an overlapping manner, determining a current view angle based on the sensor data to switch or update the template, acquiring a corresponding view angle image when a preset acquisition condition is met according to the matching degree of the preview image and the template and in combination with the angle relationship, and performing image acquisition. And repeatedly forming a multi-view image sequence and generating interactive panoramic display. According to the method, collection standardization is realized through template guidance, angle and matching joint constraint, sequence consistency is improved, and continuous interactive vehicle watching is supported.

Description

Technical Field

[0001] This application relates to the field of computer image processing technology, and in particular to a method and system for panoramic acquisition and display of vehicle exterior. Background Technology

[0002] With the development of mobile internet and online transactions, the selection, comparison, and consultation of vehicles, especially used cars, are gradually shifting from offline stores to online platforms. In order to reduce users' decision-making costs and enhance the intuitiveness of remote car viewing, there is a growing demand in the industry to showcase vehicle exteriors in richer ways, such as through multi-angle pictures, short videos, or panoramic views to present vehicle exterior details, so that users can understand the vehicle's condition as comprehensively as possible without being physically present.

[0003] In related technologies, vehicle exterior displays are still primarily based on two-dimensional static images. Typically, inspectors or staff take several pictures from fixed, experienced angles and then upload them to a platform for display. This method is simple and low-cost, but due to the lack of standardized angles and composition during the shooting process, the image quality is greatly affected by factors such as operator experience, positioning, shooting distance, and lighting conditions. This can easily lead to issues like vehicle subject misalignment, inconsistent scale, incomplete angle coverage, and discontinuous adjacent viewpoints, making it difficult for users to form a holistic understanding of the vehicle's appearance. On the other hand, to achieve a more realistic "walk-around" effect, existing technologies also employ dedicated acquisition equipment such as turntables / multi-camera arrays or 3D modeling / panoramic stitching. However, these solutions often require additional hardware investment or complex acquisition and processing procedures, placing high demands on the shooting location and operational standards. They are difficult to implement stably in large-scale, dispersed mobile acquisition scenarios, and the consistency and interactive continuity of the generated display content can still be affected by fluctuations in data acquisition quality.

[0004] Therefore, in the online panoramic display of vehicle exteriors and mobile image acquisition, the lack of standardization in shooting angles and composition, the strong dependence of acquisition quality on operator experience and environmental conditions, and the difficulty in forming a continuous and interactive 360° vehicle viewing effect based on non-standard multi-angle images have become urgent problems to be solved. Summary of the Invention

[0005] This application provides a method and system for panoramic acquisition and display of vehicle exterior images, aiming to solve the problems in the existing technology of online panoramic display and mobile image acquisition of vehicle exterior images, such as the lack of standardization of shooting angles and composition, the strong dependence of acquisition quality on operator experience and environmental conditions, and the difficulty in forming a continuous and interactive 360° vehicle viewing effect based on non-standard multi-angle images.

[0006] Firstly, a method for capturing and displaying a panoramic view of a vehicle's exterior, the method comprising:

[0007] Image data of the vehicle is acquired through a mobile terminal, and attitude / angle related sensor data of the mobile terminal are also acquired.

[0008] Based on the image data, vehicle type information is determined, and a set of multi-view acquisition templates corresponding to the vehicle type is determined according to the vehicle type information. The set of multi-view acquisition templates includes multiple view templates and view angle information corresponding to each view template.

[0009] The target view template is overlaid in the viewfinder preview interface of the mobile terminal, and the current view angle is determined based on the attitude / angle related sensor data, so as to switch or update the target view template in the multi-view acquisition template set according to the current view angle;

[0010] Based on the degree of matching between the current preview image and the target view template, and combined with the angular relationship between the current view angle and the view angle information, the mobile terminal is controlled to acquire the view image corresponding to the target view template when the preset acquisition conditions are met;

[0011] Repeat the step of overlaying the target view template in the viewfinder preview interface of the mobile terminal to obtain a multi-view image sequence, and generate an interactive panoramic display of the vehicle's appearance based on the multi-view image sequence.

[0012] Optionally, in the above scheme, determining vehicle type information based on the image data includes: performing vehicle target detection and classification on image frames in the viewfinder preview image stream, and outputting vehicle type information and / or the target area location information of the vehicle in the image frame.

[0013] In the above scheme, optionally, the multi-view acquisition template set is stored and managed by the server, and the mobile terminal requests the multi-view acquisition template set from the server according to the vehicle type information.

[0014] In the above scheme, optionally, the view template is a template image with transparency information, and the overlay display includes overlaying the template image onto the view preview screen in a semi-transparent manner.

[0015] In the above scheme, optionally, the attitude / angle related sensor data includes gyroscope data; determining the current viewing angle includes: calculating the angle change based on the angular velocity output by the gyroscope, and determining the current viewing angle using the attitude at the start of the acquisition as an initial reference.

[0016] Optionally, in the above scheme, the angle relationship includes: the current viewing angle is within the preset angle tolerance range of the viewing angle information corresponding to the target viewing angle template.

[0017] In the above scheme, optionally, the degree of matching is obtained through at least one of the following methods:

[0018] Contour matching is performed based on the contour information of the vehicle target area and the contour information of the target view template.

[0019] Feature point matching is performed based on feature points in the vehicle target region and feature points in the target view template.

[0020] The preset acquisition conditions include a matching degree greater than or equal to a preset threshold.

[0021] In the above scheme, optionally, the multi-view acquisition template set includes a preset number of view templates; the preset number is 16, and the view angle information corresponding to each view template is a preset angle sequence.

[0022] Optionally, in the above scheme, generating an interactive panoramic display based on the multi-view image sequence includes:

[0023] The multi-view image sequence is uploaded to the server, which performs vehicle body segmentation processing on the multi-view image sequence to obtain a foreground mask or foreground image, and performs blurring and / or replacement processing and / or uniform filling processing on the background area based on the foreground mask to obtain the processed multi-view image sequence.

[0024] The mobile terminal loads the processed multi-view image sequence and switches the display of the corresponding view image in the processed multi-view image sequence according to user interaction.

[0025] Secondly, a vehicle exterior panoramic acquisition and display system, the system including a mobile terminal and a server;

[0026] The mobile terminal includes a camera, a posture / angle related sensor, a display, a processor, and a memory. The memory stores instructions that, when executed by the processor, cause the mobile terminal to perform the steps of the method described in the first aspect.

[0027] The server includes a template management module and an image processing module. The template management module is used to provide a set of multi-view acquisition templates corresponding to vehicle types. The image processing module is used to perform vehicle body segmentation and / or background area processing on the multi-view image sequence and provide the processed multi-view image sequence to the mobile terminal.

[0028] Compared with the prior art, this application has at least the following beneficial effects:

[0029] Based on further analysis and research of existing technical problems, this application recognizes that existing technologies in online panoramic display and mobile image acquisition of vehicle exteriors suffer from issues such as a lack of standardization in shooting angles and composition, a strong dependence on operator experience and environmental conditions on acquisition quality, and difficulty in forming a continuous and interactive 360° vehicle viewing effect based on non-standard multi-angle images. This application addresses these problems by simultaneously acquiring vehicle image data and attitude / angle-related sensor data via a mobile terminal, and determining vehicle type information based on the image data to identify a set of multi-view acquisition templates corresponding to that vehicle type. Therefore, the acquisition process is constrained from the outset by the correspondence between "vehicle type, template set, and viewing angle information," ensuring a reusable and consistent viewing angle benchmark for different vehicles during acquisition. Next, the target viewing angle template is overlaid on the preview interface, and the current viewing angle is determined based on sensor data to switch or update the target viewing angle template accordingly. This allows the system to guide the shooting process's positioning and viewing angle selection in real time to the target viewing angle that matches the template angle information.

[0030] This solution considers both the "matching degree between the current preview image and the target view template" and the "angular relationship between the current view angle and the view angle information" when acquiring view images. It only acquires the corresponding view image when preset acquisition conditions are met. This demonstrates that the acquisition action is constrained not only by the angle but also by the consistency between the preview image and the template, reducing fluctuations in multi-view image quality caused by differences in operator experience, compositional shifts, or incomplete angle coverage. Repeatedly executing template guidance and conditional acquisition under these constraints and controls yields a multi-view image sequence with clear view relationships. When generating an interactive panoramic display of the vehicle's exterior based on this sequence, user interaction allows switching between corresponding view images within the sequence, enabling continuous view browsing and supporting online "walk-around vehicle viewing" interactive displays. Therefore, this solution addresses the issues in the background technology, such as "lack of standardization in shooting angles and compositions, acquisition quality relying on operator experience, and difficulty in forming continuous interactive 360° vehicle viewing displays," providing corresponding technical effect derivations. Attached Figure Description

[0031] Figure 1 This is a flowchart illustrating a method for capturing and displaying a panoramic view of a vehicle's exterior, provided in one embodiment of this application. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0033] Currently, the industry commonly uses a two-dimensional image display solution for online showcasing of used car exteriors. This solution typically requires inspectors to manually take static photos of the vehicle from specific angles (such as the front, 45° angle, etc.) based on their personal experience. These images are then processed by the backend system and displayed to the user in static, tiled, or list format. This display only allows users to see the vehicle's exterior from that single angle, preventing them from seeing a more comprehensive view of the used car's exterior details. Furthermore, taking these few exterior photos relies entirely on the inspector's personal experience, with no standardized shooting guidelines.

[0034] To address the aforementioned issues, in this embodiment, as follows: Figure 1 As shown, a method for panoramic acquisition and display of vehicle exterior is provided, including the following steps:

[0035] Image data of the vehicle is acquired through a mobile terminal, and attitude / angle related sensor data of the mobile terminal are also acquired.

[0036] Based on the image data, vehicle type information is determined, and a set of multi-view acquisition templates corresponding to the vehicle type is determined according to the vehicle type information. The set of multi-view acquisition templates includes multiple view templates and view angle information corresponding to each view template.

[0037] The target view template is overlaid in the viewfinder preview interface of the mobile terminal, and the current view angle is determined based on the attitude / angle related sensor data, so as to switch or update the target view template in the multi-view acquisition template set according to the current view angle;

[0038] Based on the degree of matching between the current preview image and the target view template, and combined with the angular relationship between the current view angle and the view angle information, the mobile terminal is controlled to acquire the view image corresponding to the target view template when the preset acquisition conditions are met;

[0039] Repeat the step of overlaying the target view template in the viewfinder preview interface of the mobile terminal to obtain a multi-view image sequence, and generate an interactive panoramic display of the vehicle's appearance based on the multi-view image sequence.

[0040] In one implementation, the mobile terminal can be a mobile phone, tablet, or portable device equipped with a camera and inertial sensors. The mobile terminal enters a viewfinder preview mode via the camera, continuously acquiring image data of the vehicle. This image data can be image frames from a preview image stream and / or still photos obtained after the user triggers a capture. Simultaneously, the mobile terminal acquires sensor data from attitude / angle-related sensors, which can include at least one of a gyroscope, accelerometer, and electronic compass. The sensor data can be sampled at a preset frequency and recorded with timestamps.

[0041] In one implementation, the mobile terminal determines vehicle type information based on image data. Vehicle type information may be a vehicle category (e.g., SUV, sedan, MPV, etc.) and / or a model series identifier. The mobile terminal determines a set of multi-view acquisition templates corresponding to the vehicle type based on the vehicle type information; the template set may be pre-set locally or retrieved from a server, and the template set contains multiple view templates and their corresponding view angle information. The view angle information may be defined according to the angle of the vehicle's rotation around its vertical axis, for example, with the initial angle at the start of acquisition as 0°, and the remaining view angles as a preset angle sequence.

[0042] In one implementation, the mobile terminal overlays a target viewpoint template onto the viewfinder preview screen. The overlay can display at least one of the following on the preview screen: a semi-transparent template outline, a vehicle frame, composition reference lines, and a target area proportion indicator. The mobile terminal can also determine the current viewpoint angle based on attitude / angle-related sensor data, for example, through sensor fusion or by calculating the current angle based on the output of a single sensor; then, it switches or updates the target viewpoint template in the template set according to the current viewpoint angle, ensuring that the template overlaid on the preview screen corresponds to the current shooting angle.

[0043] In one implementation, the mobile terminal, based on the matching degree between the current preview image and the target viewpoint template, and combined with the angular relationship between the current viewpoint angle and viewpoint angle information, controls the mobile terminal to acquire a viewpoint image corresponding to the target viewpoint template when preset acquisition conditions are met. The preset acquisition conditions may include at least two of the following: the angular relationship meets a preset tolerance range, the matching degree reaches a preset threshold, the preview image remains stable for a certain duration, and the vehicle target area is located within a designated display area. Once the conditions are met, the mobile terminal can automatically trigger a photo capture or prompt the user to press a button to capture a photo, and associate and store the acquired viewpoint image with the corresponding viewpoint identifier.

[0044] In one implementation, the mobile terminal repeatedly executes the process of overlaying and displaying target viewpoint templates, switching or updating the templates according to the current viewpoint angle, and acquiring viewpoint images when conditions are met, until a multi-viewpoint image sequence is obtained. The multi-viewpoint image sequence can be stored in angular order or by template number. Subsequently, the mobile terminal generates an interactive panoramic display of the vehicle's exterior based on the multi-viewpoint image sequence. For example, in the display interface, the target angle index is determined according to the user's sliding / dragging direction and amplitude, and the corresponding image is selected from the multi-viewpoint image sequence for display, realizing continuous viewpoint switching of the vehicle's exterior.

[0045] This embodiment combines "vehicle type-driven multi-view template set" with "sensor angle-driven template switching / updating", and further controls the acquisition of view images with preset acquisition conditions of "template matching degree and angle relationship", so that the multi-view data acquisition has unified angle and composition constraints, forming a multi-view image sequence with a clear order, thereby supporting the interactive panoramic display of the vehicle appearance.

[0046] In this embodiment, determining vehicle type information based on the image data includes: performing vehicle target detection and classification on image frames in the viewfinder preview image stream, and outputting vehicle type information and / or the target area location information of the vehicle in the image frame.

[0047] In one implementation, vehicle type information can be determined based on image data by real-time analysis of the viewfinder preview image stream on the mobile terminal. The mobile terminal samples image frames from the preview image stream (e.g., N frames per second or at fixed intervals), performs vehicle target detection and classification on each image frame, and outputs vehicle type information.

[0048] In one implementation, vehicle target detection and classification can be achieved by an edge-side model, such as using a target detection network to output vehicle target regions (bounding boxes or segmented regions) and simultaneously output category labels; alternatively, a two-stage approach can be used: first, vehicle target regions are detected, and then the vehicle target regions are classified to obtain the vehicle type. For terminals with limited computing power, image frames can be scaled, cropped, or quantized for inference to reduce the computational load.

[0049] In one implementation, the target region location information of the vehicle in the image frame includes at least one of: bounding box coordinates, segmentation mask, and keypoint coordinates. This target region location information can be used to determine the ROI region during subsequent template matching calculations, or to display the vehicle's location box, alignment prompts, etc., in the preview interface.

[0050] This embodiment performs vehicle target detection and classification on image frames in the preview image stream, and outputs vehicle type information and vehicle target area location information, so that template selection and subsequent matching calculation have clear target objects and ROI basis, thereby providing usable input support for the multi-view acquisition process.

[0051] In this embodiment, the multi-view acquisition template set is stored and managed by the server, and the mobile terminal requests the multi-view acquisition template set from the server according to the vehicle type information.

[0052] In this embodiment, the view template is a template image with transparency information, and the overlay display includes overlaying the template image onto the view preview screen in a semi-transparent manner.

[0053] In one implementation, the viewpoint template is a template image with transparency information. The template image can be an image format containing an alpha channel, such as PNG, or generated by combining a template outline with transparency parameters. The template image may include elements such as the vehicle's outer outline, a reference boundary for the vehicle's body proportions, and markers indicating the shooting position.

[0054] In one implementation, the overlay display includes superimposing a template image onto the viewfinder preview screen in a semi-transparent manner. The mobile terminal can perform alpha blending in the graphics rendering pipeline: using the preview screen as the base image and the template image as the overlay layer, the display intensity of the template is controlled by an opacity coefficient. The overlay display can be refreshed in real time as the template is switched to keep the overlay content consistent with the current target viewpoint.

[0055] In one implementation, the overlay display can also be combined with the vehicle target area location information to align the template position. For example, the reference center point of the template can be aligned with the center point of the vehicle detection frame, or the template can be scaled proportionally according to the size of the detection frame and then overlaid to adapt to scale changes caused by different shooting distances.

[0056] This embodiment provides a visual composition reference for the shooting process by displaying a template image with transparency information in a semi-transparent overlay on the viewfinder preview screen, so that the captured images have a consistent alignment basis in terms of composition position and scale.

[0057] In this embodiment, the attitude / angle related sensor data includes gyroscope data; determining the current viewing angle includes: calculating the angle change based on the angular velocity output by the gyroscope, and determining the current viewing angle using the attitude at the start of the acquisition as an initial reference.

[0058] In one implementation, the attitude / angle-related sensor data includes gyroscope data. The gyroscope can output angular velocities about various axes, and the mobile terminal selects the axis relevant to the vehicle's surround shooting (e.g., the yaw axis about the vertical direction) as the angle calculation input.

[0059] In one implementation, determining the current viewing angle includes calculating the angle change based on the angular velocity output by the gyroscope. Specifically, the angular velocity can be integrated over the sampling period to obtain the angle increment, and multiple angle increments can be accumulated to obtain the relative angle change. To reduce the influence of noise, the angular velocity can be low-pass filtered or smoothed using a sliding window before integration.

[0060] In one implementation, the current viewing angle is determined using the initial posture at the start of acquisition as an initial reference. The mobile terminal records the initial angle reference (e.g., defined as 0°) at the start of entering the acquisition mode, and the subsequent current viewing angle is obtained by superimposing the relative changes; alternatively, the angle reference can be reset or calibrated after each target viewing angle acquisition is completed to adapt to different acquisition strategies.

[0061] This embodiment introduces gyroscope data and calculates the angle change based on angular velocity. It obtains the current viewpoint angle by collecting the initial attitude as an initial reference, providing angle input for template switching / updating and angle relationship judgment.

[0062] In this embodiment, the angle relationship includes: the current viewing angle is within the preset angle tolerance range of the viewing angle information corresponding to the target viewing angle template.

[0063] In this embodiment, the degree of matching is obtained through at least one of the following methods:

[0064] Contour matching is performed based on the contour information of the vehicle target area and the contour information of the target view template.

[0065] Feature point matching is performed based on feature points in the vehicle target region and feature points in the target view template.

[0066] The preset acquisition conditions include a matching degree greater than or equal to a preset threshold.

[0067] In this embodiment, the multi-view acquisition template set includes a preset number of view templates; the preset number is 16, and the view angle information corresponding to each view template is a preset angle sequence.

[0068] In this embodiment, generating an interactive panoramic display based on the multi-view image sequence includes:

[0069] The multi-view image sequence is uploaded to the server, which performs vehicle body segmentation processing on the multi-view image sequence to obtain a foreground mask or foreground image, and performs blurring and / or replacement processing and / or uniform filling processing on the background area based on the foreground mask to obtain the processed multi-view image sequence.

[0070] The mobile terminal loads the processed multi-view image sequence and switches the display of the corresponding view image in the processed multi-view image sequence according to user interaction.

[0071] In this embodiment, the gyroscope is a sensor used on a mobile phone to measure or maintain direction and angular velocity; the inspector is an operator who uses a mobile terminal to collect vehicle image data.

[0072] In one embodiment, a 360° panoramic vehicle viewing method is provided, including the following steps:

[0073] Step 1: Obtain vehicle model data:

[0074] First, a lightweight vehicle recognition model based on the YOLO architecture, deployed on a mobile device, is used to process video frames in the camera preview stream in real time. The model can quickly identify the basic vehicle type (such as SUV, sedan, MPV, etc.) and output structured data.

[0075] Step 2: Obtain the template image for the current vehicle model:

[0076] Based on the vehicle model recognition result, the system requests a set of standardized multi-angle shooting templates (e.g., 16 images) matching the vehicle model from the cloud server via network. After receiving the template images, the client processes them using the alpha channel to achieve a semi-transparent effect and overlays them as a visual guide layer on top of the real-time camera preview.

[0077] Step 3: The phone's gyroscope automatically determines the corresponding shooting angle.

[0078] Using the gyroscope sensor built into the mobile phone, the system calculates the yaw angle change of the terminal device around the Z-axis (vertical axis) in real time through an integral algorithm. Taking an initial shooting position as a 0° reference, when the device rotates to a preset standard angle (such as 22.5°, 45°, etc.), the system automatically triggers a UI prompt and switches to display a semi-transparent visual template for the corresponding angle.

[0079] Step 4: Automatic shooting by comparing images and templates:

[0080] During filming, dual conditions are determined in real time in parallel:

[0081] Angle condition: Whether the current device angle calculated by the gyroscope is within the preset standard angle tolerance range.

[0082] Composition conditions: The matching degree between the vehicle outline in the current frame of the camera and the superimposed semi-transparent template is calculated in real time through image contour matching or feature point comparison algorithms.

[0083] The system will only automatically trigger the camera to take and save images when both the angle and composition conditions (e.g., a similarity of more than 90%) are met, ensuring that each photo is captured at the correct angle and composition. This process will automatically loop until images from all 16 angles have been captured.

[0084] Step 5: Process the photos and generate a 360° rotatable vehicle display:

[0085] The 16 collected standard images were uploaded to the server. The server utilized a more powerful vehicle recognition and image segmentation model to identify the subject in each image and performed background blurring or replacement to highlight the vehicle. The processed high-quality image sequence was then sent to the user's device. The front-end application loaded this image sequence and combined it into a 360° interactive rotating view that supports user gestures and dragging.

[0086] This embodiment uses a dual guidance and automatic shooting mechanism combining gyroscope and visual template to transform the non-standard manual shooting process into a standardized process executed by machines, fundamentally eliminating the differences in human operation and ensuring the absolute consistency and high quality of the source data.

[0087] The resulting 360° interactive view provides users with an immersive "virtual walk around the car" experience, offering far more information than traditional flat images. Users can freely and continuously observe every detail of the vehicle's exterior, greatly enhancing the realism of online car viewing and decision-making efficiency.

[0088] This embodiment first uses a lightweight object detection model (YOLO) to identify vehicle types in real time and automatically loads corresponding multi-angle (e.g., 16) standard shooting templates. During shooting, the system innovatively fuses mobile phone gyroscope sensor data with visual templates in real time: the gyroscope accurately determines whether the photographer has moved to a preset standard angle (e.g., every 22.5°) using an angle integration algorithm, solving the problem of "where to shoot"; simultaneously, the semi-transparent visual templates floating on the camera interface provide the photographer with real-time composition references, solving the problem of "how to shoot". This combination of gyroscope physical positioning and visual composition guidance transforms complex professional photography specifications into a simple, intuitive, and user-friendly set of operating instructions.

[0089] As the inspector moves around the vehicle, the system performs a dual-condition check in parallel: the first is the angle condition, where the gyroscope determines whether the current shooting angle has reached the standard position; the second is the composition condition, where a real-time image processing algorithm calculates the degree of conformity between the vehicle's outline in the current camera frame and the floating template. Only when both conditions are met simultaneously (i.e., at the correct angle, and the conformity of the composition with the standard template exceeds a preset threshold, such as 90%), will the system automatically trigger the camera to take a picture. This dual-verification mechanism completely eliminates the subjectivity and uncertainty of manual operation, ensuring that every captured image is taken at a precise angle and with perfect composition.

[0090] In one embodiment, a vehicle exterior panoramic acquisition and display system is provided, including a mobile terminal and a server.

[0091] The mobile terminal includes a camera, a posture / angle related sensor, a display, a processor, and a memory. The memory stores instructions that, when executed by the processor, cause the mobile terminal to perform the steps of the method described in the above embodiments.

[0092] The server includes a template management module and an image processing module. The template management module is used to provide a set of multi-view acquisition templates corresponding to vehicle types. The image processing module is used to perform vehicle body segmentation and / or background area processing on the multi-view image sequence and provide the processed multi-view image sequence to the mobile terminal.

[0093] In one embodiment, the vehicle exterior panoramic acquisition and display system includes a mobile terminal and a server. The mobile terminal includes a camera, attitude / angle related sensors, a display, a processor, and a memory; the processor executes instructions in the memory to implement the method flow, the camera and sensors provide image data and angle related data input, and the display is used to display the view preview, template overlay, and panoramic interactive interface.

[0094] In one implementation, the server includes a template management module and an image processing module. The template management module maintains a set of multi-view acquisition templates corresponding to different vehicle types and responds to mobile terminal requests by returning template resources and view angle information. The image processing module receives multi-view image sequences uploaded by the mobile terminal, performs vehicle body segmentation, and performs blurring / replacement / uniform filling and other processing on the background area to generate a processed multi-view image sequence and provide it to the mobile terminal.

[0095] In one implementation, the mobile terminal and the server communicate via a network, which can be a cellular network, Wi-Fi, or other data network. The mobile terminal can use a caching strategy to store the template set and the processed image sequence, while the server can use task queues and parallel processing strategies to improve the throughput of segmentation and background processing.

[0096] This embodiment deploys the acquisition capabilities on a mobile terminal that includes a camera and attitude / angle-related sensors, and deploys template management and image processing capabilities on the server side, thereby achieving a systematic collaboration of multi-view template-guided acquisition, sequence processing, and interactive display. This enables the method described in the above embodiment to be implemented in a combination of hardware and software.

[0097] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A method for panoramic acquisition and display of vehicle exterior, characterized in that, The method includes: Image data of the vehicle is acquired through a mobile terminal, and attitude / angle related sensor data of the mobile terminal are also acquired. Based on the image data, vehicle type information is determined, and a set of multi-view acquisition templates corresponding to the vehicle type is determined according to the vehicle type information. The set of multi-view acquisition templates includes multiple view templates and view angle information corresponding to each view template. The target view template is overlaid in the viewfinder preview interface of the mobile terminal, and the current view angle is determined based on the attitude / angle related sensor data, so as to switch or update the target view template in the multi-view acquisition template set according to the current view angle; Based on the degree of matching between the current preview image and the target view template, and combined with the angular relationship between the current view angle and the view angle information, the mobile terminal is controlled to acquire the view image corresponding to the target view template when the preset acquisition conditions are met; Repeat the step of overlaying the target view template on the viewfinder preview interface of the mobile terminal to obtain a multi-view image sequence, and generate an interactive panoramic display of the vehicle's appearance based on the multi-view image sequence.

2. The method according to claim 1, characterized in that, The step of determining vehicle type information based on the image data includes: performing vehicle target detection and classification on image frames in the viewfinder preview image stream, and outputting vehicle type information and / or the target area location information of the vehicle in the image frame.

3. The method according to claim 1, characterized in that, The multi-view acquisition template set is stored and managed by the server, and the mobile terminal requests the multi-view acquisition template set from the server according to the vehicle type information.

4. The method according to claim 1, characterized in that, The view template is a template image with transparency information, and the overlay display includes overlaying the template image onto the view preview screen in a semi-transparent manner.

5. The method according to claim 1, characterized in that, The attitude / angle related sensor data includes gyroscope data; Determining the current viewing angle includes: calculating the angle change based on the angular velocity output by the gyroscope, and using the attitude at the start of the acquisition as an initial reference to determine the current viewing angle.

6. The method according to claim 1, characterized in that, The angle relationship includes: the current view angle is within the preset angle tolerance range of the view angle information corresponding to the target view template.

7. The method according to claim 1, characterized in that, The degree of matching is obtained through at least one of the following methods: Contour matching is performed based on the contour information of the vehicle target area and the contour information of the target view template. Feature point matching is performed based on feature points in the vehicle target region and feature points in the target view template. The preset acquisition conditions include a matching degree greater than or equal to a preset threshold.

8. The method according to claim 1, characterized in that, The multi-view acquisition template set contains a preset number of view templates; the preset number is 16, and the view angle information corresponding to each view template is a preset angle sequence.

9. The method according to claim 1, characterized in that, The generation of an interactive panoramic display based on the multi-view image sequence includes: The multi-view image sequence is uploaded to the server, which performs vehicle body segmentation processing on the multi-view image sequence to obtain a foreground mask or foreground image, and performs blurring and / or replacement processing and / or uniform filling processing on the background area based on the foreground mask to obtain the processed multi-view image sequence. The mobile terminal loads the processed multi-view image sequence and switches the display of the corresponding view image in the processed multi-view image sequence according to user interaction.

10. A vehicle exterior panoramic acquisition and display system, characterized in that, Including mobile terminals and server-side components; The mobile terminal includes a camera, a posture / angle related sensor, a display, a processor, and a memory. The memory stores instructions that, when executed by the processor, cause the mobile terminal to perform the steps performed by the mobile terminal in the method as described in any one of claims 1 to 9. The server includes a template management module and an image processing module. The template management module is used to provide a set of multi-view acquisition templates corresponding to vehicle types. The image processing module is used to perform vehicle body segmentation and / or background area processing on the multi-view image sequence and provide the processed multi-view image sequence to the mobile terminal.

Images