Panoramic VR (virtual reality) and offline retail combined technical service platform

Abstract

The invention discloses a panoramic VR and offline retail combined technical service platform. The technical service platform comprises the following steps of panoramic image capture, virtual shopping guide system, interactive experience, data analysis and multi-platform compatibility. The method has the beneficial effects that the technical service platform combines a panoramic VR technology with offline retail to provide immersive shopping experience for consumers, and the users can enter virtual shops through VR equipment, browse commodities and in-shop environments in a 360-degree panoramic manner, even interact with virtual shopping guide and understand commodity details and sales promotion information, so that the shopping experience of the consumers is improved. A panoramic VR technology is combined with an offline retail system, a brand new shopping mode is provided, through personalized recommendation and immersive experience, the shopping experience and satisfaction of a user are enhanced, retailers are helped to optimize operation management, improve the sales conversion rate and expand the market range, the system has remarkable commercial value, the user stickiness and the shopping experience are enhanced, and the shopping experience of the retailers is improved. And retailers are helped to stand out in fierce market competition.

Description

Technical Field

[0001] The present application relates to a technical service platform, specifically a technical service platform combining panoramic VR with offline retail, belonging to the field of panoramic VR, retail technology and e-commerce application technology. Background Art

[0002] With the rapid development of virtual reality technology, VR is being used more and more widely in various industries. The offline retail industry is facing fierce market competition and the challenges of digital transformation. Panoramic VR technology provides a new solution for it. By improving customer experience, increasing sales opportunities and optimizing store management, panoramic VR technology has demonstrated great application potential in the retail industry.

[0003] Panoramic VR is a real-scene virtual reality technology based on panoramic images. It is a very core part of virtual reality technology. Panorama is a group or multiple groups of photos taken by a camera in a 360° circle stitched into a panoramic image, and a real-scene restoration and display method for all-round interactive viewing is achieved through computer technology. When using panoramic VR for offline retail, it is not easy to guide shopping based on user information, which reduces the user experience. In addition, offline shopping is relatively inconvenient and it is difficult to conduct virtual shopping at any location. Summary of the invention

[0004] The purpose of this application is to provide a technical service platform combining panoramic VR with offline retail in order to solve the above problems.

[0005] The present application achieves the above-mentioned purpose through the following technical solutions: a technical service platform combining panoramic VR with offline retail, the technical service platform includes the following steps:

[0006] (1) Panoramic image capture: Use high-precision panoramic camera equipment to capture the entire store and generate high-quality 360-degree panoramic images;

[0007] (2) Virtual shopping guide system: A virtual shopping guide that integrates AI technology can recommend products based on the user's browsing behavior and interests and provide personalized shopping suggestions;

[0008] (3) Interactive experience: Users can enter the virtual store through VR devices or mobile devices to view products, try them on, and perform other interactive operations;

[0009] (4) Data analysis: By collecting and analyzing user behavior data in virtual stores, it helps retailers optimize product placement and promotion strategies;

[0010] (5) Multi-platform compatibility: The technology supports a variety of VR devices and mobile applications, allowing users to conveniently conduct virtual shopping at home or anywhere else.

[0011] In the technical solution, the step (1) is equipped with a high-precision panoramic camera device, such as a multi-lens panoramic camera (such as Insta360 Pro 2) or a combination of multiple single-lens cameras to ensure that all angles of the physical store are covered. The camera device has high resolution, high dynamic range (HDR) and good low-light performance to ensure image quality.

[0012] In the technical solution, in step (1), panoramic image stitching software is developed, and images taken from multiple shooting angles are seamlessly stitched into a complete 360-degree panoramic image using image stitching algorithms (such as SIFT, SURF, etc.). Automatic color correction and exposure adjustment algorithms are used to ensure color consistency and exposure uniformity of the stitched image.

[0013] In this technical solution, the AI ​​technology is integrated in step (2), and deep learning algorithms (such as CNN, RNN, etc.) are used to analyze the user's browsing behavior and build a recommendation system model. By training the neural network, the virtual shopping guide can provide personalized product recommendations, realize the natural language processing (NLP) function, support the voice interaction between the user and the virtual shopping guide, and improve the user experience; based on the user's historical browsing and purchase records, personalized product recommendations are provided, and the reinforcement learning algorithm is used to dynamically adjust the recommendation strategy to improve the accuracy of the recommendation. Users can view and manage their shopping history records, which is convenient for searching and re-purchasing. Shopping lists, wish lists and other functions are provided to improve the user experience. .

[0014] In this technical solution, in step (3), a virtual store application is developed using engines such as Unity or Unreal Engine, which supports various interactive operations of users in the virtual store, and uses gesture recognition technology and motion tracking technology to enhance the user's sense of immersion; users can try on clothing, home furnishings and other products in a virtual environment, experience a more intuitive shopping process, and combine augmented reality (AR) technology to achieve a more realistic try-on effect. Users can share their virtual shopping experience on social platforms and interact with friends. Support real-time sharing and virtual shopping parties to increase the fun and social nature of shopping.

[0015] In this technical solution, a big data analysis platform is established in step (4), and tools such as Hadoop or Spark are used to collect and analyze user behavior data in real time. Through machine learning algorithms (such as K-means clustering, regression analysis, etc.), the product placement and promotion strategies are optimized, and visualization tools (such as Tableau and Power BI) are used to display the analysis results, helping retailers to more intuitively understand user behavior and market trends; real-time analysis of user behavior data in virtual stores, generating behavior analysis reports, and helping retailers optimize product display and marketing strategies. Cluster analysis and association rule mining are used to discover user preferences and shopping patterns, and market trends are analyzed through big data to predict popular products and user needs, provide decision support for retailers, and combine external data sources (such as social media data) to conduct more comprehensive market analysis.

[0016] In the technical solution, in step (1), SIFT (Scale-Invariant Feature Transform) or SURF (Speeded-Up Robust Features) algorithm is used to extract feature points from multiple pictures taken, RANSAC (RANdom SAmple Consensus) algorithm is used to match feature points in different images, and mismatched points are eliminated. The matched images are seamlessly spliced ​​using gradient domain fusion technology to generate a complete 360-degree panoramic image. A multi-band fusion algorithm (Multi-Band Blending) can also be used to optimize the splicing effect.

[0017] In the present technical solution, in step (2), the user's browsing behavior data is analyzed through a deep learning algorithm (such as LSTM, GRU), the user's interest points are predicted, and collaborative filtering algorithms (such as user-based collaborative filtering, item-based collaborative filtering) and content recommendation algorithms (such as TF-IDF, Word2Vec) are used to recommend products that may be of interest to the user. Combined with matrix decomposition technology (such as SVD, ALS), the recommendation accuracy is further improved; the user can consult the virtual shopping guide for product information, and the virtual shopping guide will answer based on the information in the database, supporting multiple consultation methods such as text, voice, and video to improve interactivity. The virtual shopping guide will actively push current promotional activities and discount information to the user, and use A / B testing to optimize the push strategy to ensure the effectiveness of information push.

[0018] In the technical solution, in step (3), a deep learning model (such as DeepSpeech, Wav2Vec) is used for speech recognition to convert the user's voice input into text, and pre-trained language models such as BERT and GPT are used to understand the user's semantics and generate corresponding response content, combined with a dialogue management system to process complex user queries.

[0019] In the present technical solution, in step (5), compatible VR head-mounted display devices, such as Oculus Rift, HTC Vive, etc., are prepared to ensure that users can use the virtual shopping platform smoothly. In addition, portable devices, such as a smartphone and a VR box, are also required to facilitate users to experience it anytime and anywhere.

[0020] The beneficial effects of this application are: this technical service platform provides consumers with an immersive shopping experience by combining panoramic VR technology with offline retail. Users can enter virtual stores through VR devices, browse products and store environments in 360 degrees, and even interact with virtual shopping guides to understand product details and promotional information. This not only improves consumers' shopping experience, but also helps retailers increase user stickiness and conversion rate. Combining panoramic VR technology with offline retail systems provides a new way of shopping. Through personalized recommendations and immersive experience, it enhances users' shopping experience and satisfaction, helps retailers optimize operations management, improves sales conversion rate, and expands market scope. It has significant commercial value. The technology can be applied to brand flagship stores, large shopping malls, pop-up stores, overseas shopping and other scenarios, and has broad application potential in the future, such as customized shopping experience, virtual social shopping, virtual reality marketing activities and cross-industry applications. Through the implementation of the above technical solutions, the intelligent service platform can effectively improve the digitalization level of offline retail, enhance user stickiness and shopping experience, and help retailers stand out in the fierce market competition. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a flowchart of the technical service platform for this application. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0023] A technical service platform combining panoramic VR with offline retail, the technical service platform includes the following steps:

[0024] (1) Panoramic image capture: Use high-precision panoramic camera equipment to capture the entire store and generate high-quality 360-degree panoramic images;

[0025] (2) Virtual shopping guide system: A virtual shopping guide that integrates AI technology can recommend products based on the user's browsing behavior and interests and provide personalized shopping suggestions;

[0026] (3) Interactive experience: Users can enter the virtual store through VR devices or mobile devices to view products, try them on, and perform other interactive operations;

[0027] (4) Data analysis: By collecting and analyzing user behavior data in virtual stores, it helps retailers optimize product placement and promotion strategies;

[0028] (5) Multi-platform compatibility: The technology supports a variety of VR devices and mobile applications, allowing users to conveniently conduct virtual shopping at home or anywhere else.

[0029] In the technical solution, the step (1) is equipped with a high-precision panoramic camera device, such as a multi-lens panoramic camera (such as Insta360 Pro 2) or a combination of multiple single-lens cameras to ensure that all angles of the physical store are covered. The camera device has high resolution, high dynamic range (HDR) and good low-light performance to ensure image quality.

[0030] In the technical solution, in step (1), panoramic image stitching software is developed, and images taken from multiple shooting angles are seamlessly stitched into a complete 360-degree panoramic image using image stitching algorithms (such as SIFT, SURF, etc.). Automatic color correction and exposure adjustment algorithms are used to ensure color consistency and exposure uniformity of the stitched image.

[0031] In this technical solution, the AI ​​technology is integrated in step (2), and deep learning algorithms (such as CNN, RNN, etc.) are used to analyze the user's browsing behavior and build a recommendation system model. By training the neural network, the virtual shopping guide can provide personalized product recommendations, realize the natural language processing (NLP) function, support the voice interaction between the user and the virtual shopping guide, and improve the user experience; based on the user's historical browsing and purchase records, personalized product recommendations are provided, and the reinforcement learning algorithm is used to dynamically adjust the recommendation strategy to improve the accuracy of the recommendation. Users can view and manage their shopping history records, which is convenient for searching and re-purchasing. Shopping lists, wish lists and other functions are provided to improve the user experience. .

[0032] In this technical solution, in step (3), a virtual store application is developed using engines such as Unity or Unreal Engine, which supports various interactive operations of users in the virtual store, and uses gesture recognition technology and motion tracking technology to enhance the user's sense of immersion; users can try on clothing, home furnishings and other products in a virtual environment, experience a more intuitive shopping process, and combine augmented reality (AR) technology to achieve a more realistic try-on effect. Users can share their virtual shopping experience on social platforms and interact with friends. Support real-time sharing and virtual shopping parties to increase the fun and social nature of shopping.

[0033] In this technical solution, a big data analysis platform is established in step (4), and tools such as Hadoop or Spark are used to collect and analyze user behavior data in real time. Through machine learning algorithms (such as K-means clustering, regression analysis, etc.), the product placement and promotion strategies are optimized, and visualization tools (such as Tableau and Power BI) are used to display the analysis results, helping retailers to more intuitively understand user behavior and market trends; real-time analysis of user behavior data in virtual stores, generating behavior analysis reports, and helping retailers optimize product display and marketing strategies. Cluster analysis and association rule mining are used to discover user preferences and shopping patterns, and market trends are analyzed through big data to predict popular products and user needs, provide decision support for retailers, and combine external data sources (such as social media data) to conduct more comprehensive market analysis.

[0034] In the technical solution, in step (1), SIFT (Scale-Invariant Feature Transform) or SURF (Speeded-Up Robust Features) algorithm is used to extract feature points from multiple pictures taken, RANSAC (RANdom SAmple Consensus) algorithm is used to match feature points in different images, and mismatched points are eliminated. The matched images are seamlessly spliced ​​using gradient domain fusion technology to generate a complete 360-degree panoramic image. A multi-band fusion algorithm (Multi-Band Blending) can also be used to optimize the splicing effect.

[0035] In the present technical solution, in step (2), the user's browsing behavior data is analyzed through a deep learning algorithm (such as LSTM, GRU), the user's interest points are predicted, and collaborative filtering algorithms (such as user-based collaborative filtering, item-based collaborative filtering) and content recommendation algorithms (such as TF-IDF, Word2Vec) are used to recommend products that may be of interest to the user. Combined with matrix decomposition technology (such as SVD, ALS), the recommendation accuracy is further improved; the user can consult the virtual shopping guide for product information, and the virtual shopping guide will answer based on the information in the database, supporting multiple consultation methods such as text, voice, and video to improve interactivity. The virtual shopping guide will actively push current promotional activities and discount information to the user, and use A / B testing to optimize the push strategy to ensure the effectiveness of information push.

[0036] In the technical solution, in step (3), a deep learning model (such as DeepSpeech, Wav2Vec) is used for speech recognition to convert the user's voice input into text, and pre-trained language models such as BERT and GPT are used to understand the user's semantics and generate corresponding response content, combined with a dialogue management system to process complex user queries.

[0037] In the present technical solution, in step (5), compatible VR head-mounted display devices, such as Oculus Rift, HTC Vive, etc., are prepared to ensure that users can use the virtual shopping platform smoothly. In addition, portable devices, such as a smartphone and a VR box, are also required to facilitate users to experience it anytime and anywhere.

[0038] It is obvious to those skilled in the art that the present application is not limited to the details of the exemplary embodiments described above, and that the present application can be implemented in other specific forms without departing from the spirit or essential features of the present application. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the present application is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present application. Any figure mark in the claims should not be regarded as limiting the claim involved.

[0039] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A technical service platform combining panoramic VR with offline retail, featuring: The technical service platform includes the following steps: (1) Panoramic image capture: Use high-precision panoramic camera equipment to capture the entire store and generate high-quality 360-degree panoramic images; (2) Virtual shopping guide system: A virtual shopping guide that integrates AI technology can recommend products based on the user's browsing behavior and interests and provide personalized shopping suggestions; (3) Interactive experience: Users can enter the virtual store through VR devices or mobile devices to view products, try them on, and perform other interactive operations; (4) Data analysis: By collecting and analyzing user behavior data in virtual stores, it helps retailers optimize product placement and promotion strategies; (5) Multi-platform compatibility: The technology supports a variety of VR devices and mobile applications, allowing users to conveniently conduct virtual shopping at home or anywhere else.

2. The panoramic VR combined with offline retail technical service platform according to claim 1 is characterized by: In step (1), a high-precision panoramic camera device is used, such as a multi-lens panoramic camera (such as Insta360 Pro 2) or a combination of multiple single-lens cameras to ensure that all angles of the physical store are covered. The camera device has high resolution, high dynamic range (HDR) and good low-light performance to ensure image quality.

3. The panoramic VR combined with offline retail technical service platform according to claim 1 is characterized by: In the step (1), panoramic image stitching software is developed, and images taken from multiple shooting angles are seamlessly stitched into a complete 360-degree panoramic image using an image stitching algorithm (such as SIFT, SURF, etc.), and automatic color correction and exposure adjustment algorithms are used to ensure color consistency and exposure uniformity of the stitched image.

4. The technical service platform for panoramic VR combined with offline retail according to claim 1, characterized in that: In step (2), AI technology is integrated, and deep learning algorithms (such as CNN, RNN, etc.) are used to analyze the user's browsing behavior and build a recommendation system model. By training the neural network, the virtual shopping guide can provide personalized product recommendations, realize the natural language processing (NLP) function, support voice interaction between users and virtual shopping guides, and improve user experience; based on the user's historical browsing and purchase records, personalized product recommendations are provided, and reinforcement learning algorithms are used to dynamically adjust the recommendation strategy to improve the accuracy of the recommendation. Users can view and manage their shopping history records, which is convenient for searching and re-purchasing. Shopping lists, wish lists and other functions are provided to improve user experience.

5. The technical service platform combining panoramic VR and offline retail according to claim 1, characterized in that: In step (3), a virtual store application is developed using engines such as Unity or Unreal Engine, which supports various interactive operations of users in the virtual store, and uses gesture recognition technology and motion tracking technology to enhance the user's sense of immersion; users can try on clothing, home furnishings and other products in a virtual environment, experience a more intuitive shopping process, and combine augmented reality (AR) technology to achieve a more realistic try-on effect. Users can share their virtual shopping experience on social platforms and interact with friends. Support real-time sharing and virtual shopping parties to increase the fun and social nature of shopping.

6. The panoramic VR combined with offline retail technical service platform according to claim 1, characterized in that: In step (4), a big data analysis platform is established, and tools such as Hadoop or Spark are used to collect and analyze user behavior data in real time. Through machine learning algorithms (such as K-means clustering, regression analysis, etc.), product placement and promotion strategies are optimized, and visualization tools (such as Tableau, Power BI) are used to display analysis results, so as to help retailers understand user behavior and market trends more intuitively. Real-time analysis of user behavior data in virtual stores, generating behavior analysis reports, helping retailers optimize product display and marketing strategies. Use cluster analysis and association rule mining to discover user preferences and shopping patterns, analyze market trends through big data, predict popular products and user needs, provide decision support for retailers, and combine external data sources (such as social media data) to conduct more comprehensive market analysis.

7. The technical service platform combining panoramic VR and offline retail according to claim 1, characterized in that: In the step (1), SIFT (Scale-Invariant Feature Transform) or SURF (Speeded-Up Robust Features) algorithm is used to extract feature points from multiple captured images, RANSAC (RANdom SAmple Consensus) algorithm is used to match feature points in different images, and mismatched points are removed. The matched images are seamlessly spliced ​​using gradient domain fusion technology to generate a complete 360-degree panoramic image. A multi-band fusion algorithm (Multi-Band Blending) can also be used to optimize the splicing effect.

8. The technical service platform combining panoramic VR and offline retail according to claim 1, characterized in that: In step (2), the user's browsing behavior data is analyzed by deep learning algorithms (such as LSTM, GRU), the user's points of interest are predicted, and collaborative filtering algorithms (such as user-based collaborative filtering, item-based collaborative filtering) and content recommendation algorithms (such as TF-IDF, Word2Vec) are used to recommend products that the user may be interested in. The recommendation accuracy is further improved by combining matrix decomposition techniques (such as SVD, ALS); Users can consult the virtual shopping guide for product information, and the virtual shopping guide will answer based on the information in the database. It supports multiple consultation methods such as text, voice, and video to enhance interactivity. The virtual shopping guide will proactively push current promotional activities and discount information to users, and use A / B testing to optimize push strategies to ensure the effectiveness of information push.

9. The technical service platform combining panoramic VR and offline retail according to claim 1, characterized in that: In step (3), a deep learning model (such as DeepSpeech, Wav2Vec) is used for speech recognition to convert the user's voice input into text. Pre-trained language models such as BERT and GPT are used to understand the user's semantics and generate corresponding response content. The dialog management system is combined to process complex user queries.

10. The technical service platform combining panoramic VR and offline retail according to claim 1, characterized in that: In step (5), compatible VR head-mounted display devices, such as Oculus Rift, HTC Vive, etc., are prepared to ensure that users can use the virtual shopping platform smoothly. In addition, portable devices are also required, such as a combination of a smartphone and a VR box, so that users can experience it anytime and anywhere.

Images