Physical store shopping guide method and shopping guide device, and computer readable storage medium

Abstract

The invention provides a shopping guide method and a shopping guide device for a physical store, and a computer readable storage medium. The shopping guide method comprises the steps that obtained visual information, voice information, individual attribute information and environment information of a customer and attribute information of multiple commodities in a physical store are input into a trained shopping guide model, and the shopping guide model is a deep learning model based on a multi-modal large language model; controlling the shopping guide model to determine a target recommendation demand subject and a target demand of the target recommendation demand subject, and determining a target recommendation commodity corresponding to the target demand; guiding the client to walk from the current position to the position of the target recommended commodity; wherein the individual attribute information at least comprises part or all of information of age, gender, height, figure and dressing style, and the environment information at least comprises part or all of information of city, weather condition and season where the physical store is located. According to the embodiment of the invention, the intelligence and experience of shopping can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a shopping guide method and a shopping guide device for a physical store, and a computer-readable storage medium. Background Art

[0002] With the rapid development of artificial intelligence and deep learning technology, more and more online intelligent shopping guide systems have emerged. However, the current physical store intelligent shopping guide systems mostly rely on a single modality (such as text, voice or visual information). When dealing with diverse user needs and complex scenarios, multiple rounds of interaction are required to confirm user needs, which often has limited effects. Compared with manual shopping guides, the intelligent cross-functional experience is poor. Summary of the invention

[0003] The present application provides a shopping guide method and a shopping guide device for a physical store, and a computer-readable storage medium, which are helpful to solve the problems of poor intelligence and poor user experience of the current smart shopping guide system for physical stores. The following is an introduction to various aspects involved in the present application.

[0004] In the first aspect, the present application provides a shopping guide method for a physical store, comprising: inputting the acquired visual information, voice information, individual attribute information, environmental information of the customer, and attribute information of multiple commodities in the physical store into a trained shopping guide model, wherein the shopping guide model is a deep learning model based on a multimodal large language model; controlling the shopping guide model to determine a target recommendation demand subject and a target demand of the target recommendation demand subject, and determining a target recommended commodity corresponding to the target demand; guiding the customer from a current location to a location where the target recommended commodity is located; wherein the individual attribute information includes at least part or all of the information on age, gender, height, body shape, and clothing style, the environmental information includes at least part or all of the information on the city where the physical store is located, weather conditions, and seasons, and the attribute information of the multiple commodities includes location information of the multiple commodities.

[0005] In a second aspect, the present application provides a shopping guide device for a physical store, comprising: an input module, used to input the acquired customer's visual information, voice information, individual attribute information, environmental information, and attribute information of multiple commodities in the physical store into a trained shopping guide model, wherein the shopping guide model is a deep learning model based on a multimodal large language model; a model processing module, used to control the shopping guide model to determine a target recommendation demand subject and a target demand of the target recommendation demand subject, and determine a target recommended commodity corresponding to the target demand; a guidance module, used to guide the customer from a current location to the location of the target recommended commodity; wherein the individual attribute information includes at least part or all of the information on age, gender, height, body shape, and clothing style, the environmental information includes at least part or all of the information on the city where the physical store is located, weather conditions, and seasons, and the attribute information of the multiple commodities includes location information of the multiple commodities.

[0006] In a third aspect, the present application provides a shopping guide device for a physical store, the shopping guide device comprising: a memory for storing code; a processor connected to the memory, for executing the code stored in the memory, so that the shopping guide device executes the shopping guide method as described in the first aspect.

[0007] In a fourth aspect, the present application provides a non-volatile computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, it can implement the method described in the first aspect.

[0008] In a fifth aspect, the present application provides a computer program product, which, when executed on an electronic device, enables the electronic device to implement the method described in the first aspect.

[0009] The embodiment of the present application is based on a multimodal large language model, which can combine the customer's personal attribute information, environmental information, voice information, and visual information to determine the customer's target needs, and match the corresponding target recommended products in combination with the customer's target needs and the information of multiple products in the physical store, and then guide the customer to the location of the target recommended products for trial wearing and trial operation. The multimodal large language model combines natural language processing with the processing capabilities of other modalities (such as text, images, audio, video, etc.), and can process and generate content in multiple formats and parse the user's intentions. The embodiment of the present application can provide customers with accurate and personalized target product recommendation services, guide the entire shopping process, and help improve the intelligence and experience of shopping. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in describing the embodiments of the present application are briefly introduced below.

[0011] Figure 1 It is a flowchart of the shopping guide method for a physical store provided in an embodiment of the present application.

[0012] Figure 2 yes Figure 1 A schematic diagram of the data flow of the shopping guide method shown.

[0013] Figure 3 It is a schematic diagram of the composition of a shopping guide device for a physical store provided in an embodiment of the present application.

[0014] Figure 4 It is a schematic diagram of component units / partial component units of another shopping guide device for a physical store provided in an embodiment of the present application. DETAILED DESCRIPTION

[0015] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The same or similar reference numerals are used in the drawings to represent the same or similar modules. It should be understood that the drawings are only schematic, and the scope of protection of the present application is not limited thereto. First, the application scenarios involved in the embodiments of the present application are introduced.

[0016] With the rapid development of artificial intelligence and deep learning technologies, more and more intelligent shopping guide systems have emerged. However, the current intelligent shopping guide systems in physical stores mostly rely on a single modality (such as text, voice or visual information). When dealing with diverse user needs and complex scenarios, multiple rounds of interaction are required to confirm user needs, which often has limited effects and is less intelligent than manual shopping guides. It can be seen that how to efficiently integrate information from multiple modalities to provide customers with more accurate and intelligent shopping recommendations and the entire shopping guide process is currently a major problem in physical store shopping.

[0017] Therefore, it is necessary to design a technical solution that can integrate information from multiple modalities to provide intelligent shopping guides for customers.

[0018] Based on this, the embodiment of the present application proposes a shopping guide method for a physical store, which can combine the user's personal attributes, environmental information, voice information and visual features to achieve efficient and personalized product recommendations and physical store shopping guides. Figure 1 The shopping guide method of the physical store in the embodiment of the present application is described in detail. Figure 1 As shown, the shopping guide method for a physical store in an embodiment of the present application may mainly include steps S110 to S130, and these steps are described in detail below.

[0019] It should be pointed out that the size of the serial numbers of the steps in the embodiments of the present application does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0020] In step S110, the acquired visual information, voice information, individual attribute information, environmental information of the customer, and attribute information of multiple commodities in the physical store are input into the trained shopping guide model, which is a deep learning model based on a multimodal large language model.

[0021] Among them, individual attribute information includes at least part or all of the information on age, gender, height, body shape, and clothing style; environmental information includes at least part or all of the information on the city where the physical store is located, weather conditions, and seasons; and attribute information of multiple commodities includes location information of multiple commodities.

[0022] The visual information of the customer can be obtained through the camera unit (camera) to perform face recognition. For example, the camera unit obtains the user's picture, identifies the user, and confirms whether the user is a new customer or an old customer (returning customer). If the user is identified as a new customer, the preset new customer template is activated to welcome the user. If the user is identified as an old customer, the preset old customer template is activated to communicate. The user's voice information can be obtained through the microphone.

[0023] The location information of the multiple commodities refers to the location information of the multiple commodities in the current physical store. The physical store may be, for example, a clothing store, a shoe store, a comprehensive shopping mall, etc.

[0024] Large language models (LLMs) refer to deep learning models trained with large amounts of text data that can generate natural language text or understand the meaning of language text. With the rapid advancement of information technology and the explosive growth of data volume, the capabilities of single-modal systems are no longer sufficient to cope with complex real-world tasks. Multimodal large language models (MLLMs) combine natural language processing with the processing capabilities of other modalities (such as text, images, audio, video, etc.), and can process and generate content in multiple formats. They are complex artificial intelligence systems used to process and integrate various types of data. Through MLLMs, users' personal attributes, environmental information, voice input, and visual feature information can be comprehensively analyzed to parse the user's intentions.

[0025] like Figure 2As shown, the shopping guide device 300 of the physical store may include a speech recognition module, a large language model agent and a visual language model (VLM). The speech recognition module is used to recognize the user's voice information. The visual language model is a multimodal model that can learn from images and texts at the same time. The input is images and texts, and the output is texts. The visual language model can be, for example, a general meaning.

[0026] It is understandable that before applying the shopping guide model, the shopping guide model is first trained. In some implementations, multiple samples of the training set and the attribute information of multiple commodities in the physical store are input into the initial shopping guide model, and the shopping guide model is a deep learning model based on a multimodal large language model. Any sample includes the customer's visual information, voice information, individual attribute information, and environmental information. Based on multiple target outputs and objective functions, the initial shopping guide model is trained. The multiple target outputs include at least a target recommendation demand subject, a target demand of the target recommendation demand subject, and a target recommended commodity corresponding to the target demand. Among them, the individual attribute information includes at least part or all of the information of age, gender, height, body shape, and dressing style; the environmental information includes at least part or all of the information of the city where the physical store is located, the weather conditions, and the season; the attribute information of multiple commodities includes the location information of multiple commodities.

[0027] In step S120, the shopping guide model is controlled to determine the target recommendation demand subject and the target demand of the target recommendation demand subject, and determine the target recommended commodity corresponding to the target demand.

[0028] Use a multimodal large language model to preprocess the images and descriptions of all products in the physical store inventory, extract features and store them in a vector database, such as FAISS or Milvus. Milvus is an open source multimodal vector retrieval engine that focuses on real-time retrieval and analysis. By converting text descriptions into vector representations, the product (target demand) described by the customer can be matched by calculating the similarity between the text and the product image.

[0029] Customers who enter physical stores sometimes buy items for themselves and sometimes for others. Therefore, the needs of the demand subject and customers should be clarified first.

[0030] In some implementations, the above-mentioned control of the shopping guide model to determine the target recommendation demand subject and the target demand of the target recommendation demand subject may include: using the shopping guide model to determine whether the customer is the target recommendation demand subject; if the customer is the target recommendation demand subject, then based on the customer's visual information, voice input information, individual attribute information and environmental information, the shopping guide model is controlled to recommend the customer's target demand; if the customer is not the target recommendation demand subject, then the shopping guide model is controlled to ask guiding questions to the customer according to a preset template to determine the target recommendation demand subject and the target demand of the target recommendation demand subject. Using the shopping guide model, the target recommendation demand subject to be purchased can be first determined, and then the corresponding target demand can be determined, and the target recommended goods can be matched based on the target demand.

[0031] If it is not certain whether the current user is the target recommendation demand subject, the large language model agent can ask the user guiding questions to determine whether the user is the target recommendation demand subject. If the user is identified as the target recommendation demand subject, the system will automatically fill in and use the user's individual attribute information to analyze the results.

[0032] When customers make inquiries and communicate through voice input, the large language model agent identifies the user's intention, that is, determines the customer's target needs. The large language model agent will combine the user's description content, perform semantic analysis, and identify key demand information. For example, when the user enters "What are the light shoes suitable for summer", the large language model agent will extract key information such as "suitable for summer" and "light shoes". You can perform associative reasoning, associate "light shoes" with additional attributes such as "breathable" and "comfortable", and use the integrated attributes to query. For example, "female", "25-30 years old", "medium height and thin", "casual style", "suitable for summer", "light shoes", "breathable", "comfortable" and other attribute words.

[0033] Physical stores usually retain the shopping history information of old customers, which can reflect the customer's shopping tendencies to a certain extent. In some implementations, if the customer is determined to be the subject of the target recommendation demand, then based on the customer's visual information, voice input information, individual attribute information, and environmental information, the shopping guide model is controlled to recommend the customer's target demand, which may include: determining whether the customer is a repurchase customer (old customer); if the customer is determined to be a repurchase customer and the subject of the target recommendation demand, then based on the customer's visual information, voice input information, individual attribute information, and environmental information, as well as the customer's historical shopping information, the shopping guide model is controlled to recommend the customer's target demand. Combined with the customer's historical shopping information and tendencies, more matching target recommended products and the latest promotional information can be recommended to the customer, which helps to improve the matching degree and further enhance the customer's shopping experience and intelligence.

[0034] In step S130, the customer is guided from the current location to the location where the target recommended product is located.

[0035] After multimodal products are matched, the location information corresponding to the product can be directly extracted. Based on the user's current location and destination (the location of the product), a path algorithm is used to calculate the shortest path or the user's preferred path.

[0036] The location information of the goods in the physical store can be displayed in the form of text, image, or video, etc. However, the text display method is not intuitive and clear enough.

[0037] In some implementations, a physical store is provided with multiple camera units, and the physical store is divided into multiple different sub-areas based on the viewing angles of the multiple camera units, and the multiple camera units correspond to the multiple sub-areas; the step of obtaining the location information of multiple commodities in the physical store may include: obtaining a first image, the first image is an image taken by a first camera unit, and the first camera unit is any one of the multiple camera units; identifying the commodities in the first image, and determining the location information of the commodities in the first image, and the commodities in the first image are some of the multiple commodities; updating the location information of the multiple commodities at a preset time interval, and the preset time interval may be, for example, 2 hours, 4 hours, 24 hours, etc. In this way, the specific location information of each commodity distributed in multiple sub-areas in the physical store can be obtained through multiple camera units, so as to locate and search for the commodities.

[0038] In some specific implementations, the viewing area of ​​each camera unit can be mapped to a specific location in the store to form a relative position network. Specifically, first make an accurate store floor plan, which can be created using CAD software or generated through indoor mapping tools. According to the viewing range of the camera unit, the floor plan is divided into multiple grids or areas, and each camera unit corresponds to an area. The specific steps of using a multimodal model to recognize the picture taken by each camera unit may include: cropping the image of each product, identifying its position on the shelf, such as the shelf and the layer, and establishing the location information of the product. Determine the correspondence between the product and the location, and for each product in the physical store (product library), generate and update the corresponding location information regularly (such as daily). For example, when products are in and out of the warehouse, the location of all products is updated.

[0039] In some embodiments, the product distribution information captured by the camera unit in the physical store can be retrieved, and the multimodal model can be used to find the product location, and guide the location of the product that the user is interested in. The path from the customer's current location to the location of the target recommended product can be planned, and the customer can be guided from the current location to the location of the target recommended product according to the planned navigation path, so as to facilitate the customer to select the physical target recommended product, or to perform operations such as trying on and trying on.

[0040] Figure 2 yes Figure 1 A schematic diagram of the data flow of a shopping method shown in FIG. Figure 2 As shown, the shopping guide device 300 of a physical store may include: a speech recognition module, a large language model agent and a visual-language model. The speech recognition module is used to recognize the user's speech information.

[0041] The physical store includes multiple camera units, which can obtain image information taken by the multiple camera units, and determine the location information of multiple commodities based on the image information taken by the multiple camera units. After determining the target demand of the target recommendation demand subject and the target recommended commodity corresponding to the target demand, the customer can be guided from the current location to the location where the target recommended commodity is located.

[0042] A physical store usually includes multiple shelves. A single shelf may include multiple layers or multiple storage locations. Multiple items are usually placed on any layer. Multiple items may block each other. The camera unit is also limited by the shooting angle. It is often difficult to accurately identify the location of all the goods only by relying on the images taken by the camera unit.

[0043] In some implementations, any product in a physical store has a corresponding identifier, and the information of the identifier includes the second location information of the product. The identifier may be, for example, a barcode or a QR code of the product. The second location information is text location information associated with the identifier. The above-mentioned identification of the product in the first image and determination of the location information of the product in the first image may include: identifying the first location information of the product in the first image, the first location information being information obtained based on a visual sensor, and the first location information may be the same as the second location information. The first location information is corrected according to the second location information (information of the identifier) ​​corresponding to the product in the first image, and the location information of the product in the first image is determined. The location information of the product in the first image may include text form and image form. Alternatively, the second location information corresponding to the product in the first image and the first location information are fused to determine the location information of the product in the first image. For example, the priority of the first location information is higher than that of the second location information, and the location information of the product in the first image may be determined according to the priority of the first location information and the second location information. The fusion process is performed in combination with the initial location information displayed in the image and the information of the identifier of the product, which helps to improve the accuracy of identifying the location information of the product.

[0044] In some implementations, within a preset time period, if the deviation between the first location information and the second location information exceeds a preset threshold, a prompt message of location divergence may be sent to the user so that the user (administrator) can make timely adjustments. The preset threshold may be, for example, an area corresponding to the same camera unit. If the deviation between the first location information and the second location information exceeds an area corresponding to the same camera unit, a prompt message of location divergence may be sent to the user.

[0045] In some implementations, before guiding the customer from the current location to the location of the target recommended product, the method of the embodiment of the present application may also include: displaying prompt information of the target recommended product, the prompt information includes at least one or more of the following information: picture information of the target recommended product, picture information of the location of the target recommended product, and voice prompt information for navigating to the location of the target recommended product. For example, displaying a picture of the target recommended product on the display screen of the shopping guide device makes it easier for the customer to see the appearance of the product in time, and displaying a picture of the location of the target recommended product makes it easier for the customer to quickly locate the location of the target recommended product. For another example, in the process of navigating to the location of the target recommended product, voice prompt information such as "Please turn left" and "Be careful of the steps" may be issued.

[0046] If the physical store is a clothing store or a shoe store, in some implementations, after guiding the customer from the current location to the location of the target recommended product, the method of the embodiment of the present application may further include: guiding the customer to a nearby fitting room or fitting place to try it on according to the type of the target recommended product. If the type of the target recommended product is a coat, the customer may be guided to a nearby fitting place to try it on. If the type of the target recommended product is underwear, the customer may be guided to a nearby fitting room to try it on, thereby satisfying the customer's privacy needs.

[0047] In some implementations, the method of the embodiment of the present application may further include: in response to a customer's instruction to settle the bill, guiding the customer to a nearby cashier counter for settlement, thereby providing customers with thoughtful and comprehensive shopping services and experiences.

[0048] The embodiment of the present application is based on a multimodal large language model, which can combine the customer's personal attribute information, environmental information, voice information, and visual information to determine the customer's target needs, and match the corresponding target recommended products in combination with the customer's target needs and the information of multiple products in the physical store, and then guide the customer to the location of the target recommended product for trial wearing and trial operation. The multimodal large language model combines natural language processing with the processing capabilities of other modalities (such as text, images, audio, video, etc.), and can process and generate content in multiple formats and parse the user's intentions. The embodiment of the present application can provide customers with accurate and personalized target product recommendation services, guide the entire shopping process, help improve the intelligence and experience of shopping, and is suitable for shopping guide scenarios in physical stores (clothing stores, shoe stores, etc.).

[0049] Combination of the above Figure 1-Figure 2 The method embodiment of the present application is described in detail. Figure 3 to Figure 4 The device embodiment of the present application is described in detail. It should be understood that the description of the device embodiment corresponds to the description of the method embodiment, so the parts not described in detail can refer to the previous method embodiment.

[0050] An embodiment of the present application provides a shopping guide device for a physical store. Figure 3 Schematic diagram of the composition of the shopping guide device of a physical store provided in the embodiment of the present application. Figure 3 As shown, the shopping guide device 300 of the physical store includes: an input module 310 , a model processing module 320 , and a guiding module 330 .

[0051] The input module 310 is used to input the acquired visual information, voice information, individual attribute information, environmental information of the customer, and attribute information of multiple commodities in the physical store into the trained shopping guide model. The shopping guide model is a deep learning model based on a multimodal large language model.

[0052] Among them, individual attribute information includes at least part or all of the information on age, gender, height, body shape, and clothing style; environmental information includes at least part or all of the information on the city where the physical store is located, weather conditions, and seasons; and attribute information of multiple commodities includes location information of multiple commodities.

[0053] The model processing module 320 is used to control the shopping guide model to determine the target recommendation demand subject and the target demand of the target recommendation demand subject, and determine the target recommended commodity corresponding to the target demand.

[0054] The guidance module 330 is used to guide the customer from the current location to the location where the target recommended product is located.

[0055] Optionally, the model processing module 320 is used to use the shopping guide model to determine whether the customer is a target recommendation demand subject; if the customer is a target recommendation demand subject, the shopping guide model is used to recommend the customer's target demand based on the customer's visual information, voice input information, individual attribute information and environmental information; if the customer is not a target recommendation demand subject, the shopping guide model asks guiding questions to the customer according to a preset template to determine the target recommendation demand subject and the target demand of the target recommendation demand subject.

[0056] Optionally, the model processing module 320 is used to determine whether the customer is a repeat customer; if the customer is determined to be a repeat customer and is a target recommendation demand subject, the shopping guide model is used to recommend the customer's target demand based on the customer's visual information, voice input information, individual attribute information and environmental information, as well as the customer's historical shopping information.

[0057] Optionally, a plurality of camera units are arranged in the physical store, and the physical store is divided into a plurality of different sub-areas based on the field of view of the plurality of camera units, and the plurality of camera units correspond to the plurality of sub-areas. The shopping guide device 300 may further include a location acquisition module, and the location acquisition module is used to acquire a first image, the first image is an image taken by a first camera unit, and the first camera unit is any camera unit among the plurality of camera units; identify the commodity in the first image, determine the location information of the commodity in the first image, and the commodity in the first image is part of the plurality of commodities; and update the location information of the plurality of commodities at a preset time interval.

[0058] Optionally, any commodity has a corresponding identifier, and the information of the identifier includes the second position information of the commodity. The identifier may be, for example, a barcode or a QR code of the commodity. The second position information is the position information associated with the identifier. The position acquisition module is used to identify the first position information of the commodity in the first image; and to correct the first position information according to the second position information corresponding to the commodity in the first image, so as to determine the position information of the commodity in the first image.

[0059] Optionally, the shopping guide device 300 may also include a display module, which is used to display prompt information of the target recommended product, and the prompt information includes at least one or more of the following information: picture information of the target recommended product, picture information of the location of the target recommended product, and voice prompt information for navigating to the location of the target recommended product.

[0060] If the physical store is a clothing store or a shoe store, optionally, the guiding module 330 is also used to guide the customer to a nearby fitting room or fitting place for trying on the goods according to the type of the target recommended goods.

[0061] Figure 4 Schematic diagram of component units / partial component units of another shopping guide device for a physical store provided in an embodiment of the present application. Figure 4 As shown, the shopping guide device 400 of a physical store may include: a memory 410 and at least one processor 420.

[0062] The memory 410 is used to store codes or computer programs.

[0063] The processor 420 is connected to the memory 410 and is used to execute the code or computer program stored in the memory 410 to control the shopping guide device 400 of the physical store to execute any of the processing methods described above.

[0064] Exemplarily, the computer program may be divided into one or more modules / units, one or more modules / units are stored in the memory 410 and executed by the processor 420 to complete the present application.

[0065] Those skilled in the art will understand that Figure 4 The shopping guide device 400 is only an example of a physical store, and does not constitute a limitation on the physical store. The shopping guide device 400 may include more or fewer components than shown in the figure, or combine some components, or different components.

[0066] The processor 420 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc.

[0067] The shopping guide device 400 for a physical store provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0068] An embodiment of the present application further provides a non-volatile computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0069] An embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0070] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above method embodiments, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above method embodiments can be implemented. Among them, the computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / electronic device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), read-only compact disc (CD-ROM), magnetic tape, floppy disk and optical data storage device. The computer-readable storage medium mentioned in the present application can be a non-volatile storage medium, in other words, it can be a non-transient storage medium.

[0071] It should be noted that the information collection process (such as face image collection process, fingerprint collection process, etc.) / feature extraction process involved in this application is performed with the user's knowledge and permission, that is, the information collection process / feature extraction process complies with the requirements of laws and regulations and does not constitute an act that harms the public interest.

[0072] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0073] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0074] In the embodiments provided in the present application, it should be understood that the disclosed devices / equipment and methods can be implemented in other ways. For example, the device / equipment embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0075] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0076] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0077] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0078] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0079] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A shopping guide method for a physical store, characterized in that: include: Input the acquired visual information, voice information, individual attribute information, environmental information of the customer, and attribute information of multiple commodities in the physical store into a trained shopping guide model, wherein the shopping guide model is a deep learning model based on a multimodal large language model; Control the shopping guide model to determine a target recommendation demand subject and a target demand of the target recommendation demand subject, and determine a target recommended commodity corresponding to the target demand; Guiding the customer from the current location to the location where the target recommended product is located; Among them, the individual attribute information at least includes part or all of the information on age, gender, height, body shape, and clothing style; the environmental information at least includes part or all of the information on the city where the physical store is located, weather conditions, and seasons; and the attribute information of the multiple commodities includes location information of the multiple commodities.

2. The shopping guide method according to claim 1, characterized in that: The controlling the shopping guide model to determine a target recommendation demand subject and a target demand of the target recommendation demand subject includes: Using the shopping guide model, determining whether the customer is a target recommendation demand subject; If the customer is the target recommendation demand subject, based on the customer's visual information, voice input information, individual attribute information and environmental information, control the shopping guide model to recommend the customer's target demand; If the customer is not the target recommendation demand subject, the shopping guide model is controlled to ask guiding questions to the customer according to a preset template to determine the target recommendation demand subject and the target demand of the target recommendation demand subject.

3. The shopping guide method according to claim 2, characterized in that: If the customer is the target recommendation demand subject, then based on the customer's visual information, voice input information, individual attribute information and environmental information, controlling the shopping guide model to recommend the customer's target demand includes: Determining whether the customer is a returning customer; If it is determined that the customer is a repeat customer and is the target recommendation demand subject, the shopping guide model is controlled to recommend the customer's target demand based on the customer's visual information, voice input information, individual attribute information and environmental information, as well as the customer's historical shopping information.

4. The shopping guide method according to claim 1, characterized in that: A plurality of camera units are arranged in the physical store, and the physical store is divided into a plurality of different sub-areas based on the viewing angle ranges of the plurality of camera units, and the plurality of camera units correspond to the plurality of sub-areas; The step of obtaining the location information of the plurality of commodities in the physical store comprises: Acquire a first image, where the first image is an image taken by the first camera unit, and the first camera unit is any camera unit among the multiple camera units; Identify commodities in the first image and determine location information of the commodities in the first image, where the commodities in the first image are part of the multiple commodities; The location information of the plurality of commodities is updated at preset time intervals.

5. The shopping guide method according to claim 4, characterized in that: Any of the commodities has a corresponding identifier, and the information of the identifier includes the second location information of the commodity; The step of identifying the commodity in the first image and determining the location information of the commodity in the first image includes: Identifying first location information of the commodity in the first image; The first position information is corrected according to the second position information corresponding to the commodity in the first image to determine the position information of the commodity in the first image.

6. The shopping guide method according to any one of claims 1 to 5, characterized in that: Before guiding the customer from the current location to the location where the target recommended product is located, the method further includes: Display prompt information of the target recommended product, wherein the prompt information includes at least one or more of the following information: Image information of the target recommended product, image information of the location of the target recommended product, and voice prompt information for navigating to the location of the target recommended product.

7. A shopping guide device for a physical store, characterized in that: include: An input module is used to input the acquired visual information, voice information, individual attribute information, environmental information of the customer, and attribute information of multiple commodities in the physical store into a trained shopping guide model, wherein the shopping guide model is a deep learning model based on a multimodal large language model; A model processing module, used for controlling the shopping guide model to determine a target recommendation demand subject and a target demand of the target recommendation demand subject, and to determine a target recommended commodity corresponding to the target demand; A guiding module, used for guiding the customer from the current location to the location of the target recommended product; Among them, the individual attribute information at least includes part or all of the information on age, gender, height, body shape, and clothing style; the environmental information at least includes part or all of the information on the city where the physical store is located, weather conditions, and seasons; and the attribute information of the multiple commodities includes location information of the multiple commodities.

8. The shopping guide device according to claim 7, characterized in that: A plurality of camera units are arranged in the physical store, and the physical store is divided into a plurality of different sub-areas based on the viewing angle ranges of the plurality of camera units, and the plurality of camera units correspond to the plurality of sub-areas; The shopping guide device also includes: A position acquisition module is used to acquire a first image, where the first image is an image taken by the first camera unit, and the first camera unit is any one of the multiple camera units; identify the goods in the first image, and determine the position information of the goods in the first image, where the goods in the first image are part of the multiple goods; and update the position information of the multiple goods at preset time intervals.

9. A shopping guide device for a physical store, characterized in that: The shopping guide device comprises: A memory for storing codes; A processor is connected to the memory and is used to execute the code stored in the memory so that the shopping guide device executes the shopping guide method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed, the computer program is used to implement the shopping guide method according to any one of claims 1 to 6.

Images