Augmented reality-based vehicle body garland display method and head-mounted display device

Abstract

The invention relates to an augmented reality-based vehicle body garland display method and a head-mounted display device.The augmented reality-based vehicle body garland display method comprises the steps of displaying a virtual interaction interface in response to a vehicle body garland setting request initiated by a user, and displaying a target vehicle model to be subjected to garland setting in the virtual interaction interface; wherein the target vehicle model is constructed on the basis of spatial data obtained after three-dimensional scanning is carried out on an entity vehicle; in response to a garland pattern input by a user, displaying a vehicle body garland preview result of the target vehicle model on the virtual interaction interface; wherein the vehicle body garland preview result is formed by collaging in a collaging area of the target vehicle model based on the pattern features of the garland pattern and the vehicle features of the target vehicle model. A virtual interaction interface based on augmented reality vehicle body garland setting can be provided for a user, and therefore real-time interaction with the vehicle body garland setting effect of the user can be achieved.

Description

Technical Field

[0001] The present application relates to the field of computer vision, and in particular to a vehicle body decal display method and a head-mounted display device based on augmented reality. Background Art

[0002] Car decals are decorative stickers that personalize and enhance a car's exterior. They can feature a variety of patterns, colors, and text, including geometric shapes, abstract art, animals, and plants. Currently, design professionals often create designs based on the car owner's needs. Once the designs are complete, they apply the decals to the actual vehicle using a collage tool. This method lacks real-time feedback on the collage effect and offers no interactive channel for users to make real-time adjustments.

[0003] There is currently no effective solution to the problem that related technologies cannot achieve real-time interaction in setting car body decals. Summary of the Invention

[0004] In this embodiment, a vehicle body decal display method and a head-mounted display device based on augmented reality are provided to solve the problem in related technologies that real-time interaction in vehicle body decal setting cannot be achieved.

[0005] In a first aspect, this embodiment provides a method for displaying vehicle body decals based on augmented reality, including:

[0006] In response to a user-initiated request to set a vehicle body decal, a virtual interactive interface is displayed, and a target vehicle model to be set with the decal is displayed in the virtual interactive interface; wherein the target vehicle model is constructed based on spatial data obtained by performing a three-dimensional scan of a physical vehicle;

[0007] In response to the decal pattern input by the user, a preview result of the body decal of the target vehicle model is displayed on the virtual interactive interface; wherein the body decal preview result is collaged in the collage area of the target vehicle model based on the pattern features of the decal pattern and the vehicle features of the target vehicle model.

[0008] In some embodiments, the method further comprises:

[0009] In response to the user's adjustment operation on the vehicle body decal in the vehicle body decal preview result, the vehicle body decal adjustment result is displayed on the virtual interactive interface.

[0010] In some embodiments, the method further comprises:

[0011] In response to the user's perspective change, a preview result of the vehicle body decal under the user's new perspective is displayed on the virtual interactive interface; wherein the perspective change is determined based on monitoring the user's head movement and eye movement.

[0012] In some embodiments, in response to the decal pattern input by the user, displaying a preview result of the decal of the target vehicle model on the virtual interactive interface includes:

[0013] In response to the decal pattern input by the user, a preview result of the body decal of the target vehicle model is displayed on the virtual interactive interface, and a material display effect and a lighting display effect are added to the body decal preview result; wherein, the material display effect is set according to the material of the physical vehicle corresponding to the target vehicle model; and the lighting display effect is set according to the surface lighting reflection information of the physical vehicle.

[0014] In some embodiments, in response to the decal pattern input by the user, displaying a preview result of the decal of the target vehicle model on the virtual interactive interface includes:

[0015] In response to the user's setting of the overlay effect of the multiple lace patterns input, the multiple lace patterns are superimposed and displayed in the vehicle body lace preview result of the virtual interactive interface according to the overlay effect set by the user; the overlay effect includes the transparency and rotation angle of each lace pattern after superposition, and the overlay order between different lace patterns.

[0016] In some embodiments, the method further comprises:

[0017] In response to the user's confirmation operation on the vehicle body decal preview result, a vehicle cover production order corresponding to the vehicle body decal preview result is output to a related order system.

[0018] In a second aspect, a method for setting a vehicle body decal based on augmented reality is provided in this embodiment, the method comprising:

[0019] Obtain the decal pattern input by the user and the target vehicle model to be set for the decal;

[0020] Performing image recognition on the garland pattern to determine pattern features of the garland pattern and vehicle features of the target vehicle;

[0021] According to the pattern features and the vehicle features, the decal pattern is tiled in the tiling area of the target vehicle model, and the pattern tiling result is output to the associated virtual interactive interface, so that the virtual interactive interface displays the vehicle body decal preview result corresponding to the target vehicle model and the pattern tiling result.

[0022] In some embodiments, the step of tiling the garland pattern on the tiling area of the target vehicle model according to the pattern features and the vehicle features includes:

[0023] According to the vehicle curved surface features represented by the target vehicle model, setting the tiling curvature of the lace pattern in the tiling area of the target vehicle model;

[0024] According to the size of the tiling area, setting the tiling size of the lace pattern in the tiling area;

[0025] According to the vehicle contour features represented by the target vehicle model, in combination with the tiling curvature, the tiling size and the pattern features, the tiling shape of the lace pattern in the tiling area is deformed.

[0026] In some embodiments, the step of tiling the garland pattern on the tiling area of the target vehicle model according to the pattern features and the vehicle features includes:

[0027] Identifying a door edge and a vehicle body interface region in the target vehicle model;

[0028] When the tiling area includes the door edge, the lace pattern is edge-aligned with the door edge according to the door edge and the pattern features, so as to perform pattern tiling of the lace pattern in the tiling area of the target vehicle model;

[0029] When the tiling area includes the vehicle body boundary area, based on a preset gradient algorithm and in combination with the pattern features, the color and / or shape of the lace pattern are gradually set to perform pattern collage on the tiling area of the target vehicle model.

[0030] In some embodiments, the step of tiling the garland pattern on the tiling area of the target vehicle model according to the pattern features and the vehicle features includes:

[0031] Upon receiving a plurality of latte art patterns input by a user, obtaining pattern features corresponding to the plurality of latte art patterns respectively based on image recognition of the latte art patterns;

[0032] Determining preset splicing modes corresponding to the plurality of latte patterns according to pattern features corresponding to the plurality of latte patterns;

[0033] The plurality of garland patterns are pattern-spliced in the collage area according to the preset splicing mode and the vehicle characteristics.

[0034] In a third aspect, a head-mounted display device is provided in this embodiment, including: a camera component, a display component, and a processor;

[0035] The camera component is used to capture the user's interaction gestures and transmit the interaction gestures to the processor;

[0036] The display component is used to execute the vehicle body decal display method based on augmented reality as described in the first aspect above;

[0037] The processor is used to execute the vehicle body decal setting method based on augmented reality as described in the second aspect above.

[0038] Compared to related technologies, this embodiment provides a vehicle body decal display method and head-mounted display device based on augmented reality. The vehicle body decal display method, in response to a user-initiated request to set a vehicle body decal, displays a virtual interactive interface and displays a target vehicle model to be set with the decal in the virtual interactive interface. The target vehicle model is constructed based on spatial data obtained from a three-dimensional scan of a physical vehicle. In response to a decal pattern input by the user, a preview of the vehicle body decal of the target vehicle model is displayed on the virtual interactive interface. The preview of the vehicle body decal is a collage of the pattern features of the decal pattern and the vehicle features of the target vehicle model, formed by collage in the collage area of the target vehicle model. This method can provide users with a virtual interactive interface for setting vehicle body decals based on augmented reality, thereby enabling real-time interaction with the user's vehicle decal setting effects.

[0039] The details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0041] Figure 1 This is a hardware structure block diagram of a terminal for the vehicle body decal display method based on augmented reality in this embodiment;

[0042] Figure 2 This is a flow chart of the vehicle body decal display method based on augmented reality in this embodiment;

[0043] Figure 3 This is a flow chart of the method for setting car body decals in this embodiment;

[0044] Figure 4 is a flow chart of a method for displaying vehicle body decals based on augmented reality in some embodiments;

[0045] Figure 5 Schematic diagram of the structure of the head-mounted display device of this embodiment. DETAILED DESCRIPTION

[0046] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0047] Unless otherwise defined, technical or scientific terms used in this application shall have the ordinary meanings as understood by persons of ordinary skill in the art to which this application belongs. The terms "a," "an," "the," "these," and similar expressions in this application do not denote limitations on quantity and may be singular or plural. The terms "comprise," "include," "have," and any variations thereof, as used in this application, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include unlisted steps or modules (units) or other steps or modules (units) inherent to the process, method, product, or device. The terms "connected," "connected," "coupled," and similar expressions used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. As used in this application, "plurality" means two or more. "And / or" describes an association between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone; A and B exist simultaneously; or B exists alone. Generally, the character " / " indicates that the objects in the preceding and following relationship are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0048] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 This is a hardware structure diagram of the terminal of the vehicle body decal display method based on augmented reality in this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 The processor 102 (only one is shown) and a memory 104 for storing data, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The terminal may also include a transmission device 106 for communication functions and an input / output device 108. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0049] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the vehicle body decal display method based on augmented reality in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implementing the above-mentioned method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0050] Transmission device 106 is used to receive or transmit data via a network. This network may include a wireless network provided by the terminal's communications provider. In one embodiment, transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0051] In this embodiment, a method for displaying car body decals based on augmented reality is provided. Figure 2 This is a flow chart of the vehicle body decal display method based on augmented reality in this embodiment. Figure 2 As shown, the process includes the following steps:

[0052] Step S210, in response to a vehicle body decal setting request initiated by the user, a virtual interactive interface is displayed, and a target vehicle model to be decal set is displayed in the virtual interactive interface; wherein the target vehicle model is constructed based on spatial data obtained after a three-dimensional scan of the physical vehicle.

[0053] The virtual interactive interface can be provided by the display component of a head-mounted display device worn by the user. The head-mounted display device can first provide an initial interactive interface based on a vehicle-related AR application. When the user clicks a button in the initial interactive interface (for example, based on user gesture recognition to determine whether the user clicked the button), a vehicle body decal setting request can be initiated, thereby entering the virtual interactive interface.

[0054] Users can enter an augmented reality (AR) environment using augmented reality (AR) devices, such as AR glasses or smart devices with AR applications installed. Within this AR environment, the system captures the real world around the user in real time, including the actual scene of the physical vehicle. Using AR technology, the system overlays a virtual preview of the vehicle's decals in the user's field of view.

[0055] After scanning the physical vehicle to be decorated, a target vehicle model corresponding to the physical vehicle can be displayed in the virtual interactive interface. Specifically, a depth camera or RGB-D camera can be used to perform a three-dimensional scan of the physical vehicle to obtain spatial data of the vehicle surface, such as position, shape, and curvature. An accurate three-dimensional model of the physical vehicle is then created and displayed as the target vehicle model in the virtual interactive interface.

[0056] Step S220, in response to the decal pattern input by the user, a preview result of the body decal of the target vehicle model is displayed on the virtual interactive interface; wherein the preview result of the body decal is based on the pattern features of the decal pattern and the vehicle features of the target vehicle model, and is collaged in the collage area of the target vehicle model.

[0057] Users can scan the desired vehicle decal using the camera on the head-mounted display, scan and upload it using other image acquisition devices, or select it from a library of decals provided by the user through a virtual interactive interface. The decal can be an existing design or a hand-drawn one. Computer vision is used to identify the decal's features and convert them into digital form.

[0058] Afterwards, the vehicle body decals can be applied to the user-defined collage area based on the vehicle features of the target vehicle model and the pattern characteristics of the decals. The collage area can be a partial area of the vehicle body or the entire vehicle body. Once the pattern collage is completed, the system uses a depth camera or RGB-D camera to collect three-dimensional spatial information of the vehicle body to accurately calculate the relationship between the vehicle surface and the virtual pattern. Based on the collage results, a preview of the vehicle body decal is generated and previewed on the virtual interactive interface. Users can preview the vehicle body decal effect in real time in the augmented reality environment and perform interactive operations to adjust the position, angle, size, etc.

[0059] A library of various design options is available for users to choose from, and new designs can be customized based on their needs. Once the user confirms the design of the vehicle body design, a message is sent to the associated order system, which generates a production order for the custom car cover based on the finalized vehicle body design preview results, facilitating order placement and production.

[0060] At the hardware level, the vehicle body decal display method provided in this embodiment can be implemented using a head-mounted display (HMD). This HMD is equipped with a camera and sensors, such as inertial sensors, motion capture sensors, and interpupillary distance correction sensors. The HMD's built-in processor or associated computer or server can support deep learning and image processing. At the software level, software modules based on computer vision and image processing can be developed, along with a deep learning framework (such as the open-source deep learning frameworks TensorFlow or PyTorch) for training and optimizing image recognition and processing models. An augmented reality platform (such as Unity or ARKit) can also be provided for real-time preview. This creates a system architecture encompassing the front-end (user interface, AR display) and back-end (pattern processing, database). Functional and user experience testing can then be conducted, and feedback from mobile phones can be used to optimize the system architecture.

[0061] From the user's perspective, the user only needs to scan and upload the decals through a head-mounted display device or other image acquisition device to input the decals. After that, the user can preview the car body decals in real time in the virtual interactive interface through the head-mounted display device. Among them, computer vision technology can be used to scan and recognize the hand-drawn patterns, textures or other materials provided by the user, and the characteristics of various patterns can be identified through machine learning models for classification and analysis. The decals are processed by deep learning algorithms to extract features, and combined with the characteristics of the vehicle itself, the decals are collaged and optimized on the car body, so that the decals can seamlessly adapt to the vehicle's appearance. Based on augmented reality technology, virtual images are combined with the real environment. Users can directly observe the car body decals in a real environment through head-mounted display devices, so as to view the effect of the car body decals in an intuitive and real-time manner.

[0062] In related technologies, creating a decal often requires significant artistic and design skills, creating a high barrier to entry for users. Existing decal creation tools also often offer limited customization options, making it difficult for users to customize their decal settings to their exact needs. Some decal design tools also have complex user interfaces and lack user-friendliness, making it difficult for users to set up decals. In related technologies, users cannot receive real-time feedback on the decal's effect while setting it up, hindering timely adjustments and optimizations.

[0063] In this regard, this embodiment, based on steps S210 to S220, allows users to intuitively preview and manipulate virtual patterns, improving the efficiency and flexibility of decal setup while reducing setup difficulty. All user operations are immediately reflected on the display screen, shortening the feedback cycle and enhancing the user experience. By capturing three-dimensional vehicle information with a depth camera or RGB-D camera, the virtual decal pattern can be more accurately matched to the vehicle surface. Traditional display solutions typically only provide flat settings and lack adaptability to complex vehicle curves. The three-dimensional modeling of this embodiment significantly improves the accuracy of vehicle decal setup. Some embodiments also provide a community sharing feature, allowing users to share and exchange their decal setups, promoting interaction and innovation. In other embodiments, artificial intelligence (AI) algorithms can be used to analyze the type, color, shape, and other attributes of the decal pattern uploaded by the user during decal setup, as well as their habits and preferences for adjusting decals on the vehicle body, to provide personalized recommendations for different users. In addition, real-time preview and feedback can also reduce errors or problems that may need to be adjusted during the body decal setting process, thereby reducing the cost of modifying and remaking the car cover, and can achieve the setting of body decals more efficiently and at a lower cost.

[0064] Therefore, through the above steps S210 to S220, in response to a user-initiated request to set a vehicle decal, a virtual interactive interface is displayed, and a target vehicle model to be set with the decal is displayed in the virtual interactive interface. The target vehicle model is constructed based on spatial data obtained through a three-dimensional scan of a physical vehicle. In response to a decal pattern input by the user, a preview of the target vehicle model's vehicle decal is displayed in the virtual interactive interface. The preview of the vehicle decal is a collage of the target vehicle model's collage area based on the decal pattern's pattern features and the target vehicle model's vehicle features. This provides the user with an augmented reality-based virtual interactive interface for setting vehicle decals, thereby enabling real-time interaction with the user's vehicle decal setting effects.

[0065] In one embodiment, the above-mentioned vehicle body decal display method may further include:

[0066] In response to a user's adjustment of the body decals in the preview results, the results of the body decal adjustments are displayed on a virtual interactive interface. Gesture recognition can be used to identify user adjustments to the position, angle, and size of the body decals in the preview results. For example, a user can change the position and shape of each decal on the vehicle body by dragging and resizing it in real time, or adjust the transparency and rotation angle of the decals, thereby achieving flexibility in the body decal settings. Specifically, users can interact with the virtual decals through gestures or a touch interface, such as zooming in, out, rotating, or changing the color, to further adjust the decals and personalize them.

[0067] In particular, in some embodiments, audio or vibration feedback can be provided to the user based on the user's adjustment operation to enhance the immersion and interactivity of the adjustment interaction operation and improve the user experience.

[0068] Additionally, in one embodiment, the vehicle body decal display method may further include:

[0069] In response to changes in the user's perspective, a preview of the vehicle body decal from the user's new perspective is displayed on the virtual interactive interface; wherein the perspective change is determined based on monitoring the user's head movement and eye movement. The user's perspective change can be monitored in real time. When the user's perspective changes, the display angle of the target vehicle model is adjusted according to the user's changed perspective, and the decal pattern is redrawn on the vehicle body at the corresponding new perspective. Specifically, the user's head movement and eye movement can be monitored by a sensor system integrated into the head-mounted display device. For example, based on the gyroscope and accelerometer, the rotation and movement of the user's head are sensed, the position and posture of the user's head are determined, and the change in the user's perspective is calculated. The image of the user's head is captured by an external camera to track the position and movement of the head in real time.

[0070] Eye tracking technology captures and analyzes the user's eye movements to monitor changes in perspective. This allows for simultaneous adjustments to the decal's display on the vehicle body. Real-time rendering technology allows the decal to appear as if it were actually applied to the vehicle, providing users with a multi-perspective, all-encompassing display of the decal.

[0071] Additionally, upon detecting a change in the user's perspective, the virtual interface displays a preview of the vehicle decal from the corresponding perspective. For example, if the user switches from viewing the side decal preview from the front to an oblique angle, the virtual interface will display the preview of the vehicle decal at an angle. Specifically, this can be achieved using Simultaneous Localization and Mapping (SLAM) technology, which can identify and track the vehicle's position and posture in real time, ensuring that the augmented content remains synchronized with the real environment.

[0072] Furthermore, in one embodiment, in response to the decal pattern input by the user, displaying a preview result of the decal of the target vehicle model on the virtual interactive interface may include:

[0073] In response to the decal pattern input by the user, a preview result of the body decal of the target vehicle model is displayed on the virtual interactive interface, and material display effects and lighting display effects are added to the body decal preview result; wherein, the material display effect is set according to the material of the physical vehicle corresponding to the target vehicle model; and the lighting display effect is set according to the surface lighting reflection information of the physical vehicle.

[0074] Specifically, real-time rendering technology can be used to apply corresponding material display effects (such as setting the displayed texture and style) and lighting display effects to the body of the body preview result according to the material (such as metal, glass, plastic, etc.) and light and shadow effects of the physical vehicle, so as to simulate the light and shadow, texture details and the reflection effect of the body surface to the ambient light in the real environment, etc., so that the body decal effect looks more realistic.

[0075] Through AR technology, users can see the car body decals in real time under different lighting and backgrounds. This immersive experience is difficult to achieve with traditional solutions, which often display them in static conditions and lack a realistic simulation of the real world. This embodiment can intelligently adjust the display of the decals using information such as captured ambient lighting, ensuring that the car body decals maintain good visual performance even in different environments, removing the limitations of static display and improving the reliability and adaptability of car body decal settings.

[0076] In particular, in some of the embodiments, a real-scene preview function can also be provided to users. Based on the real-scene preview function, users can preview the vehicle body decal effects under different lighting and environmental backgrounds to provide visual performance of the vehicle body decal patterns under various environmental and lighting conditions.

[0077] In one embodiment, in response to a decal pattern input by a user, displaying a preview of the decal pattern of the target vehicle model on the virtual interactive interface may include:

[0078] In response to the user's setting of the overlay effect of multiple input lace patterns, multiple lace patterns are superimposed and displayed in the vehicle body lace preview result of the virtual interactive interface according to the overlay effect set by the user; the overlay effect includes the transparency, rotation angle and overlay order of each lace pattern after superposition.

[0079] Users can select or upload custom decals through a virtual interactive interface, including custom graphics, text, and color combinations. Image features are extracted based on the decal's shape, size, and other attributes. The user-defined decal is then applied to the identified vehicle body features, with the decals positioned and scaled accordingly.

[0080] Users can also upload multiple decals. These designs are managed using a hierarchical structure, including foreground and background pattern priorities. For example, based on the user's selection, one portion of the decal can be used as the foreground, and another portion as the background, allowing the decals to be overlaid. Planar and perspective transformation algorithms can be used to ensure that the decals align with the vehicle's curved surface during collage, achieving a natural visual blend. For collages of multiple decals, users can choose to overlay and interact. Overlay effects include transparency, rotation angle, and the order in which different decals are overlaid. Users can set the transparency, rotation angle, and overlay order of each decal. The decals are then displayed after being overlaid based on the user's settings. Pixel-based masking can also be used to create the effect of one decal being "inlaid" within another. This embodiment enables the overlay and display of vehicle decals, providing users with a richer and more flexible range of vehicle decal settings.

[0081] In addition, in one embodiment, the above-mentioned vehicle body decal display method may further include:

[0082] In response to the user confirming the preview result of the vehicle decal, a vehicle cover production order corresponding to the preview result is output to the associated order system. Specifically, after the user confirms the current vehicle decal setting effect, the corresponding vehicle cover production order is output to the associated order system based on the final confirmed preview result of the vehicle decal, so that a vehicle cover can be produced for the user's physical vehicle.

[0083] In this embodiment, a method for setting vehicle body decals is also provided. Figure 3 This is a flow chart of the method for setting the car body pattern in this embodiment. Figure 3 As shown, the process includes the following steps:

[0084] Step S301: Obtain the decal pattern input by the user and the target vehicle model on which the decal is to be set.

[0085] Step S302: performing image recognition on the garland pattern to determine the pattern features of the garland pattern and the vehicle features of the target vehicle.

[0086] Computer vision and deep learning algorithms can be used to detect and analyze vehicle image data to identify key vehicle features, including doors, windows, roof, body sides, and wheels. A convolutional neural network (CNN) model is trained on vehicle images from different angles to accurately identify the vehicle's geometry and key components from a variety of viewing angles and lighting conditions. The vehicle recognition results are saved as a structured data model, including the location and shape parameters of each feature, providing reference coordinates for subsequent pattern tiling. Based on the vehicle's 3D spatial information, a 3D target vehicle model can be displayed in a virtual interactive structure.

[0087] Based on the user-uploaded decal pattern, image recognition is performed based on its attributes (such as shape and size). Based on the decal pattern's characteristics, the decal pattern is tiled onto the identified vehicle body features, with the tiles in different locations adjusted and scaled. Optionally, the decal pattern's pattern features may include shape, color, texture, edge, pattern symmetry, pattern composition, repeated elements, and pattern complexity. More specifically, shape features may include basic geometric shapes such as circles, squares, triangles, and combinations of different shapes. Color features may include the color with the largest area in the decal pattern and its corresponding color value, including color gradient and contrast. Texture features may include the surface texture of the decal pattern, such as roughness, smoothness, or stripes, to help simulate the decal pattern's visual impact and texture depth. Edge features include edge position, curvature, and sharpness, to more accurately interpret the decal pattern's overall shape. The decal pattern's symmetry characterizes whether the pattern is symmetrical and the type of symmetry, such as axial or central symmetry, to provide a sense of visual balance. The composition of a latte art pattern characterizes the relative positions and arrangement of its elements, thereby conveying the overall layout of the pattern. Identifying repeating elements within a latte art pattern can aid in analyzing its repetitiveness and periodicity. The complexity of the latte art pattern, such as the number of elements and the richness of detail, can further inform the identification of its details.

[0088] By identifying the pattern features of latte art, the pattern can be converted into a processable digital format based on feature recognition, allowing for subsequent editing, application, and display. By identifying user-uploaded latte art patterns, the system can more accurately understand the user's individual needs, facilitating the provision of customized latte art setup suggestions. Furthermore, quickly and accurately identifying latte art pattern features can help improve the fluidity of interaction and reduce the difficulty of processing user-submitted latte art patterns.

[0089] Furthermore, before pattern tiling, the vehicle's features must be identified, including the location of various surface components and the vehicle's shape parameters. Shape parameters can include: contour parameters such as bounding boxes and curve outlines; key point coordinates such as key points and midpoints; dimensional ratios such as aspect ratios and size parameters; shape descriptions such as geometric shape type and angle orientation; curvature values and geometric bending parameters; texture features, surface properties (smoothness, roughness), and lighting and shading parameters.

[0090] Bounding boxes describe the outer boundaries of various vehicle parts (such as the roof, body, and doors), represented by rectangular boxes, to help define feature regions. The contour curves of various vehicle components, such as door edges, window edges, and body contours, are typically represented using mathematical equations (such as Bezier curves or B-splines). The specific coordinates (key point locations) of vehicle features (such as door handles, window corners, and lights) are typically expressed in two-dimensional coordinates to facilitate accurate positioning during subsequent decal pattern tiling. Some decal patterns may need to be tiled between different vehicle features, and midpoints can help accurately define the layout. Obtaining the aspect ratio of each vehicle feature, such as the ratio of doors to windows, provides a basis for pattern scaling and adaptation. Extracting specific dimensions (such as width, height, and depth) of vehicle features such as wheels, windows, and lights facilitates accurate matching during tiling. Geometric shape types, such as circle, rectangle, and polygon, describe the shape of vehicle features, helping the system adapt the pattern. Angle and direction features characterize the angle and direction of vehicle features relative to the overall vehicle geometry, especially for the pattern design of connecting parts. Curvature values are used to describe the curvature information of key vehicle components, such as the degree of curve of the roof and body edges, which helps to deform the decals on the curved surface. Geometric bending parameters describe the bending properties of different vehicle parts (such as local depressions or ridges) to ensure that the decals can be naturally presented on complex body curves. Rendering parameters such as texture and surface features can represent the surface characteristics of different parts of the vehicle (such as smoothness and roughness) to provide a reference for the material and color selection of the pattern. Lighting and shadow parameters are used to describe the performance of vehicle component features under different lighting conditions, helping to ensure the visual effects of the pattern in different environments.

[0091] Step S303, based on the pattern features and vehicle features, the decal pattern is tiled in the tile area of the target vehicle model, and the pattern tile result is output to the associated virtual interactive interface, so that the virtual interactive interface displays the vehicle body decal preview result corresponding to the target vehicle model and the pattern tile result.

[0092] After extracting pattern features, the decal can be applied to different vehicle models based on feature processing, simulating different styles and effects. Based on the identified pattern features, AI technology can also be used to analyze and recommend similar design elements or styles, expanding design inspiration for users and automatically generating similar decal designs based on their style preferences, improving design efficiency. Before applying the decal to the vehicle body, feature extraction can be used to check its integrity and quality to ensure optimal display.

[0093] By extracting various vehicle shape features and constructing a structured data model, the system can effectively store and manage the identified vehicle feature information, systematizing it for quick search and access, and more accurately placing decals in the virtual display. It also supports precise tiling, for example, providing accurate coordinate and parameter information for subsequent pattern tiling to ensure a perfect fit on the vehicle body. It also helps enhance AI training data, allowing for the use of richer geometric and lighting parameters in subsequent model training, improving recognition and adaptability.

[0094] Among them, the coordinates of the decals can be mapped with the surface coordinates of the captured vehicle features to ensure that the relative position and size relationship between the two are accurate. Through coordinate mapping, when the display position of the vehicle in the virtual interactive interface moves or the display perspective rotates, the decals can be accurately matched to the specific position of the vehicle body.

[0095] Through the above steps S301 to S303, it is possible to realize the vehicle body decal setting based on augmented reality for the user, thereby realizing real-time interaction with the vehicle body decal setting effect of the user.

[0096] In one embodiment, according to the pattern features and the vehicle features, tiling the lace pattern in the tiling area of the target vehicle model may include:

[0097] According to the vehicle surface features represented by the target vehicle model, the collage curvature of the lace pattern in the collage area of the target vehicle model is set; according to the size of the collage area, the collage size of the lace pattern in the collage area is set; according to the vehicle contour features represented by the target vehicle model, combined with the collage curvature, collage size and pattern features, the collage shape of the lace pattern in the collage area is deformed.

[0098] This embodiment provides for adapting a garland pattern to the curved surface of a vehicle body. The garland pattern's shape can be adjusted based on the vehicle's curved surface features to ensure it fits snugly, enhancing the overall aesthetic of the garland pattern on the vehicle. For example, when a user scans and uploads a ring-shaped pattern as a garland pattern, the vehicle's curved surface structure is first analyzed. If a vehicle's rear windshield is curved, a deformation algorithm can be used to adjust the ring-shaped pattern into a curved arc to conform to the curve of the rear windshield, rather than forcibly applying a flat pattern to the curved surface.

[0099] In addition, this embodiment also provides for the adaptation of the garland pattern to the vehicle body size. Specifically, if the size of the garland pattern provided by the user is larger than the collage area to be tiled, an intelligent scaling algorithm can be used to adjust the proportions of the garland pattern to ensure that the garland pattern is displayed beautifully in the collage area while avoiding distortion.

[0100] Furthermore, for complex body structures like the front styling, machine learning can automatically detect the shape of the complex structure and adjust the shape of the decal accordingly. For example, stretching or compressing parts of the decal can ensure that the decal matches the streamlined contours of the vehicle body.

[0101] This embodiment can achieve surface adaptation, size adaptation, and shape deformation of the embossing pattern to the vehicle body, thereby improving the fit of the embossing pattern to the vehicle body and enhancing the user experience.

[0102] Additionally, in one embodiment, tiling the lace pattern in the tiling area of the target vehicle model may include:

[0103] Identify the door edge and the vehicle body interface area in the target vehicle model; when the collage area includes the door edge, align the edge of the lace pattern with the door edge based on the door edge and pattern features to perform a pattern collage of the lace pattern in the collage area of the target vehicle model; when the collage area includes the vehicle body interface area, based on a preset gradient algorithm and combined with the pattern features, perform a color and / or shape gradient setting of the lace pattern to perform a pattern collage of the lace pattern in the collage area of the target vehicle model.

[0104] This embodiment implements collage optimization based on edge alignment, gradient transitions, and patterned splicing. When applying a collage pattern to a car door, the door edge is identified and aligned with the edge, minimizing the gap between the collage patterns. Pixel-based masking technology allows users to achieve a more natural blend of patterns, avoiding the harsh edge cutting common with traditional methods. This enhances visual aesthetics and makes the collage more artistic and personalized.

[0105] The aforementioned vehicle body interface area can specifically be the interface between different vehicle body parts, such as the interface between the front and side panels. If the user desires to create a collage of decals at these interface areas, a gradient algorithm can be used to create a natural transition between the colors, textures, or shapes of the decals corresponding to the different vehicle body parts at these interface areas, avoiding abrupt dividing lines and improving the overall aesthetics.

[0106] In one embodiment, the graffiti pattern is applied to the collage area of the target vehicle model according to the pattern features and the vehicle features, which may specifically include:

[0107] When multiple lace patterns are received from the user, pattern features corresponding to the multiple lace patterns are obtained based on the recognition results of the lace patterns; according to the pattern features corresponding to the multiple lace patterns, a preset splicing mode corresponding to the multiple lace patterns is determined; according to the preset splicing mode and vehicle characteristics, the multiple lace patterns are pattern-spliced in the collage area.

[0108] When users upload multiple lace patterns for collage, the system analyzes the patterns' attributes, such as color, texture, and shape, and recommends a variety of preset stitching modes that match these attributes. This allows users to combine different lace patterns to achieve a unified visual effect. For example, floral patterns can be combined with geometric patterns to create a layered overall design. The specific stitching modes and the corresponding relationships between stitching modes and different pattern attributes can be customized based on the actual application scenario.

[0109] After adjusting and stitching the design, users can preview the resulting design in real time. This preview can also be displayed as a 3D simulation, allowing users to intuitively perceive how the design will appear on the vehicle and make further adjustments to achieve the optimal effect.

[0110] Next, we'll illustrate application scenarios for body decals. For example, in a racing scenario, a brand logo might need to be placed on the car's body. To allow users to quickly change the brand logo on a car's body decal, the body decal setting method provided in this embodiment can scan an existing brand logo pattern as the decal pattern, automatically adjusting the decal's shape, size, and curvature to fit the car's curved surface. Furthermore, optimized decal pattern splicing ensures a smooth and natural transition between different brand logos on the car's body, preserving the overall aesthetic.

[0111] For example, at a car show, users can create decals for their vehicles. By scanning and uploading a hand-drawn design, a connected server or computer adjusts the shape and optimizes the splicing. Users can intuitively preview the custom decal on the vehicle and flexibly modify the effect based on interaction with the head-mounted display, quickly obtaining the final result.

[0112] Figure 4 is a flow chart of a method for displaying vehicle body decals based on augmented reality in some embodiments, such as Figure 4 As shown, the vehicle body decal display method based on augmented reality includes the following steps:

[0113] Step S401: In response to a vehicle body decal setting request initiated by a user, a virtual interaction interface is displayed, and a target vehicle model to be decal-set is displayed in the virtual interaction interface.

[0114] Step S402, in response to the decal pattern input by the user, displays the body decal preview result of the target vehicle model on the virtual interactive interface, and adds material display effect and lighting display effect to the body decal preview result; wherein, the material display effect is set according to the material of the physical vehicle corresponding to the target vehicle model; and the lighting display effect is set according to the surface lighting reflection information of the physical vehicle.

[0115] Step S403: In response to the user setting the superposition effect of multiple decals, multiple decals corresponding to the superposition effect set by the user are displayed in the vehicle body decal preview result on the virtual interactive interface.

[0116] Step S404: in response to the user's adjustment operation on the vehicle body decal in the vehicle body decal preview result, the vehicle body decal adjustment result is displayed on the virtual interactive interface.

[0117] Step S405 , in response to the user's confirmation operation on the vehicle body decal preview result, outputting the vehicle body decal preview result confirmed by the user.

[0118] The above steps S401 to S405 can provide the user with a virtual interactive interface for vehicle body decal setting based on augmented reality, thereby enabling real-time interaction with the user's vehicle body decal setting effect.

[0119] In this embodiment, a head-mounted display device is also provided. Figure 5 is a structural diagram of the head mounted display device 50 of this embodiment, as shown in FIG. Figure 5 As shown, the head mounted display device 50 includes: a camera component 52, a display component 54 and a processor 56;

[0120] The camera component 52 is used to capture the user's interactive posture and transmit the interactive posture to the processor; the display component 54 is used to execute the vehicle body decal display method based on augmented reality provided in any of the above embodiments; the processor 56 is used to execute the vehicle body decal setting method based on augmented reality provided in any of the above embodiments.

[0121] It should be noted that, for specific examples in this embodiment, reference may be made to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0122] It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0123] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0124] Obviously, the accompanying drawings are merely examples or embodiments of the present application. A person skilled in the art can also apply the present application to other similar situations based on these drawings without inventive effort. Furthermore, it is understandable that, although the work involved in this development process may be complex and lengthy, certain design, manufacturing, or production changes based on the technical content disclosed in this application are merely routine technical means for a person skilled in the art and should not be considered to constitute a deficiency in the disclosure of the present application.

[0125] The term "embodiment" as used in this application refers to specific features, structures, or characteristics described in conjunction with the embodiment that can be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily mean that the embodiment is the same, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is understood, either explicitly or implicitly, by those skilled in the art that the embodiments described in this application can be combined with other embodiments when there is no conflict.

[0126] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for displaying car body decals based on augmented reality, characterized in that: include: In response to a user-initiated request to set a vehicle body decal, a virtual interactive interface is displayed, and a target vehicle model to be set with the decal is displayed in the virtual interactive interface; wherein the target vehicle model is constructed based on spatial data obtained by performing a three-dimensional scan of a physical vehicle; In response to the decal pattern input by the user, a preview result of the body decal of the target vehicle model is displayed on the virtual interactive interface; wherein the body decal preview result is collaged in the collage area of the target vehicle model based on the pattern features of the decal pattern and the vehicle features of the target vehicle model.

2. The vehicle body decal display method according to claim 1, characterized in that: The method further comprises: In response to the user's adjustment operation on the vehicle body decal in the vehicle body decal preview result, the vehicle body decal adjustment result is displayed on the virtual interactive interface.

3. The vehicle body decal display method according to claim 1, characterized in that: The method further comprises: In response to the user's perspective change, a preview result of the vehicle body decal under the user's new perspective is displayed on the virtual interactive interface; wherein the perspective change is determined based on monitoring the user's head movement and eye movement.

4. The vehicle body decal display method according to claim 1, characterized in that: In response to the decal pattern input by the user, displaying a preview result of the decal of the target vehicle model on the virtual interactive interface includes: In response to the decal pattern input by the user, a preview result of the body decal of the target vehicle model is displayed on the virtual interactive interface, and a material display effect and a lighting display effect are added to the body decal preview result; wherein, the material display effect is set according to the material of the physical vehicle corresponding to the target vehicle model; and the lighting display effect is set according to the surface lighting reflection information of the physical vehicle.

5. The vehicle body decal display method according to claim 1, characterized in that: In response to the decal pattern input by the user, displaying a preview result of the decal of the target vehicle model on the virtual interactive interface includes: In response to the user's setting of the overlay effect of the multiple lace patterns input, the multiple lace patterns are superimposed and displayed in the vehicle body lace preview result of the virtual interactive interface according to the overlay effect set by the user; the overlay effect includes the transparency and rotation angle of each lace pattern after superposition, and the overlay order between different lace patterns.

6. The vehicle body decal display method according to any one of claims 1 to 5, characterized in that: The method further comprises: In response to the user's confirmation operation on the vehicle body decal preview result, a vehicle cover production order corresponding to the vehicle body decal preview result is output to a related order system.

7. A method for setting car body decals based on augmented reality, characterized in that: The method comprises: Obtain the decal pattern input by the user and the target vehicle model to be set for the decal; Performing image recognition on the garland pattern to determine pattern features of the garland pattern and vehicle features of the target vehicle; According to the pattern features and the vehicle features, the decal pattern is tiled in the tiling area of the target vehicle model, and the pattern tiling result is output to the associated virtual interactive interface, so that the virtual interactive interface displays the vehicle body decal preview result corresponding to the target vehicle model and the pattern tiling result.

8. The method for setting car body decals according to claim 7, characterized in that: According to the pattern features and the vehicle features, the graffiti pattern is applied to the collage area of the target vehicle model, comprising: According to the vehicle curved surface features represented by the target vehicle model, setting the tiling curvature of the lace pattern in the tiling area of the target vehicle model; According to the size of the tiling area, setting the tiling size of the lace pattern in the tiling area; According to the vehicle contour features represented by the target vehicle model, in combination with the tiling curvature, the tiling size and the pattern features, the tiling shape of the lace pattern in the tiling area is deformed.

9. The method for setting car body decals according to claim 7, characterized in that: According to the pattern features and the vehicle features, the graffiti pattern is applied to the collage area of the target vehicle model, comprising: Identifying a door edge and a vehicle body interface region in the target vehicle model; When the tiling area includes the door edge, the lace pattern is edge-aligned with the door edge according to the door edge and the pattern features, so as to perform pattern tiling of the lace pattern in the tiling area of the target vehicle model; When the tiling area includes the vehicle body boundary area, based on a preset gradient algorithm and in combination with the pattern features, the color and / or shape of the lace pattern are gradually set to perform pattern collage on the tiling area of the target vehicle model.

10. The method for setting car body decals according to claim 7, characterized in that: According to the pattern features and the vehicle features, the graffiti pattern is applied to the collage area of the target vehicle model, comprising: Upon receiving a plurality of latte patterns input by a user, obtaining pattern features corresponding to the plurality of latte patterns based on image recognition of the latte patterns; Determining preset splicing modes corresponding to the plurality of latte patterns according to pattern features corresponding to the plurality of latte patterns; The plurality of garland patterns are pattern-spliced in the collage area according to the preset splicing mode and the vehicle characteristics.

11. A head-mounted display device, characterized in that: include: Camera components, display components and processors; The camera component is used to capture the user's interaction gestures and transmit the interaction gestures to the processor; The display component is used to execute the vehicle body decal display method based on augmented reality according to any one of claims 1 to 6; The processor is used to execute the vehicle body decal setting method based on augmented reality according to any one of claims 7 to 10.

Images