Sanitary appliance product identification
Through image analysis technology, multiple learning models are used to identify bathroom products, which solves the problem of difficulty in identifying bathroom appliances, improves recognition efficiency and accuracy, and reduces customer service time.
Patent Information
- Application Number
- CN202410663338.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-05-16
- Filing Date
- 2024-05-27
- Publication Date
- 2024-11-26
AI Technical Summary
In homes or buildings, it is difficult to identify bathroom appliances, especially models or serial numbers, which leads to wasted customer service time and resource consumption.
Using image-based automatic recognition technology, multiple learning models (first model, second model and third model) are used to analyze the images of the bathroom products collected by users, provide cropped image processing through parallel or serial means, and output predictions of the bathroom products. value to identify the product.
It improves the efficiency and accuracy of bathroom product identification, reduces customer service time and resource consumption, and facilitates users to self-service installation and repair.
Smart Images

Figure CN119027707A_ABST
Abstract
Description
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This application claims the benefit of and priority to U.S. Provisional Patent Application No. 63 / 468,894, filed May 25, 2023, the entire contents of which are incorporated herein by reference. Technical Field
[0003] The present disclosure relates to product identification for plumbing fixture-related devices. Background Art
[0004] Customers can contact customer service or utilize online resources to get help with a specific product. Customers can seek help with a product. For example, a customer may need help troubleshooting to determine a solution for a malfunctioning product. A customer may seek a replacement for a defective or inoperable appliance. A customer may seek a new appliance that matches the product. Unfortunately, it is not always easy for customers to identify products. Many products do not have the product name printed on them. This is especially true for plumbing fixtures or for appliances that are attached to walls, countertops, etc. in homes or buildings. Model or serial numbers can be difficult to find, may only be found on a label that has been removed, or may only be included on documentation that cannot be located. When a customer calls a customer service center or accesses online channels for these devices, one of the initial inquiries is to identify the product. This identification can involve a significant portion of customer service time and other resources.
[0005] This disclosure presents automated techniques and processes for identifying a device based on an image of the device. Summary of the Invention
[0006] According to one embodiment of the present invention, the present invention provides a method for identifying bathroom products, the method comprising: receiving an original image captured by a user, wherein the original image depicts the bathroom product; providing a first model to analyze the original image; when the first model indicates an object match for the bathroom product in the original image, providing a second model for the original image and a third model for a cropped version of the original image in parallel; when the first model lacks the object match for the bathroom product in the original image, continuously providing the second model for the original image and the third model for the cropped version of the original image; and outputting a predicted value for the bathroom product in response to the first model, the second model, and the third model.
[0007] According to another embodiment of the present invention, the present invention provides a device for identifying bathroom products, the device comprising: a communication interface configured to receive an original image of the bathroom product; a first model configured to analyze the original image of the bathroom product; a second model configured to analyze the original image of the bathroom product in combination with supplementary information of the bathroom product; a third model configured to analyze a cropped image of the bathroom product; and a controller configured to perform analysis using the first model, the second model, and the third model, wherein when the first model indicates an object match for the bathroom product in the original image, the second model and the third model are executed continuously, and when the first model lacks the object match for the bathroom product in the original image, the second model and the third model are executed in parallel.
[0008] According to another embodiment of the present invention, the present invention provides a non-volatile computer-readable medium, which includes instructions, which are configured to perform a method when executed, the method including: receiving an original image captured by a user, wherein the original image depicts a bathroom product; providing a first model to analyze the original image; when the first model indicates an object match for the bathroom product in the original image, providing a second model for the original image and a third model for a cropped version of the original image in parallel; when the first model lacks the object match for the bathroom product in the original image, continuously providing the second model for the original image and the third model for the cropped version of the original image; and outputting a predicted value for the bathroom product in response to the first model, the second model, and the third model. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] Various exemplary embodiments are described herein based on exemplary embodiments and with reference to the following drawings.
[0010] Figure 1 Illustrated is a sample customer service and product identification system.
[0011] Figure 2 Illustrated is a sample agent interface for customer service and product identification systems.
[0012] Figure 3 Illustrated is an example terminal user interface for a customer service and product identification system.
[0013] Figure 4 Illustrated is an example product identification for use in a customer service and product identification system.
[0014] Figure 5 Illustration of product images used for customer service and product identification systems.
[0015] Figure 6 Illustrated is an example learning model sequence for customer service and product recognition systems.
[0016] Figure 7 Illustrated is an example verification interface for a product identification system.
[0017] Figure 8 An example implementation is illustrated that includes multiple images of a product.
[0018] Figure 9 An example implementation is illustrated in which a partial view or exploded product is provided to a model.
[0019] Figure 10 Illustrated are example implementations in which new categories of products or categorizations of products are generated.
[0020] Figure 11 The diagram shows the Figures 1 to 10 An example controller for the example.
[0021] Figure 12 Pictured Figure 11 Flowchart of the device. DETAILED DESCRIPTION
[0022] A plumbing fixture can be defined as a device connected to the plumbing system of a house, building, or facility. A plumbing fixture assembly can include additional equipment connected to a plumbing fixture. Examples of plumbing fixtures or plumbing fixture assemblies include faucets, sinks, showerheads, urinals, toilets, or other fixtures.
[0023] Traditionally, bathroom fixtures or bathroom fixture devices require licensed plumbers to repair and / or replace. However, do-it-yourself (DIY) installation and repair are becoming more popular. Even when a professional is trying to solve a problem, it may not be obvious which product the professional is troubleshooting. Identification of bathroom fixtures is an important step for professionals. Professionals often rely on experience, knowledge and skills, but may still have difficulty identifying products. Consumers also require help with DIY installation and repair. Consumers may have multiple options to contact manufacturers or other assistance services to obtain information about bathroom fixture devices. In some cases, consumers request troubleshooting of faults in bathroom fixture devices. In some cases, consumers request assistance in selecting replacements for bathroom fixture devices. In both cases, the initial query is to identify bathroom fixture devices. The following embodiments include a system and technology for identifying bathroom fixture devices based on one or more images collected by consumers and analyzed using multiple learning models.
[0024] Although the following examples are described in the context of bathroom fixtures, the techniques and apparatus described herein are also applicable to other types of fixtures. These fixtures may be wall-mounted fixtures such as towel racks, toilet paper holders, coat hooks, lighting fixtures, mirrors, or other fixtures.
[0025] Figure 1 An example customer service and product identification system is illustrated. The system includes at least a product identification server 100 configured to send and receive data to and from an end-user device 103 via a communication session 101. The end-user device 103 may include a mobile device such as a smartphone, a computer such as a laptop, a tablet, or another electronic device. Additional, different, or fewer components may be included.
[0026] An operator, such as user 30, may operate end-user device 103. Additionally, an operator, such as agent 10, may provide commands or input data to product identification server 100. Agent 10 and user 30 may communicate via communication session 20, which may be a phone call, a chat window, or another exchange of messages, voice, or data.
[0027] The agent 10 may also communicate with the user 30 via the product identification server 100 and the end-user device 103. The communication session 101 may be generated independently of the communication session 20. For example, the communication session 101 may be conducted over a packet-switched network, and the communication session 20 may be over the public switched telephone network (PSTN) or plain old telephone service (POTS).
[0028] The communication session 101 may also be initiated by the user 30 independently of the agent 10. For example, the user 30 may visit an online web page and initiate an identification request without the assistance of an agent.
[0029] However, communication session 20 and communication session 101 can be combined into a single communication session that includes both voice and data. Examples include video calling services. Communication session 101 can be generated in response to communication session 20. That is, the conversation on communication session 20 can cause one or more parties (e.g., agent 10 or user 30) to send a link via email, text message, or other technology, so that the link is used to create communication session 101. The link can include identification information of the user, so that when the end-user device 103 executes or otherwise opens the link, the product identification server 100 receives identification information for the associated input provided by the end-user device 103 and the user 30.
[0030] For example, user 30 may initiate communication session 20 via telephone to request customer assistance regarding a bathroom product. During communication session 20, it may become apparent to communication session 20 and / or user 30 that the bathroom product cannot be adequately identified in communication session 20. In response to this determination, agent 10 may initiate communication session 101. Communication session 101 may be initiated by sending a link to the user. This link may be transmitted via communication session 20 to the same phone number used in communication session 20. This link may be sent to user 30 via email. User 30 may be instructed to download an application to establish communication session 101 or otherwise access an online hosted application. This link provides a connection for communication session 101, connecting user 30 and agent 10.
[0031] The user 30 may be prompted to provide one or more images of a bathroom product via the communication session 101. The provided images may include raw images. Raw images may be images that have not been cropped, filtered, or analyzed. The raw images may be captured by the end user device 103 (e.g., by a camera on a smartphone). The raw images are sent to the product identification server 100 via the communication session 101.
[0032] The product recognition server 100 is configured to analyze the original image to identify bathroom products. The product recognition server 100 may perform image processing techniques such as template matching, edge detection, feature extraction, or another example to identify bathroom products. The product recognition server 100 may store a set of templates, each associated with a different bathroom product. The set of templates may correspond to a product portfolio from a specific manufacturer of the product recognition server 100. Each template in the set of templates may correspond to a different SKU. The product recognition server 100 compares the original image with the set of templates and determines the specific SKU or bathroom product that best matches.
[0033] Figure 2 An example agent interface 110 for a customer service and product identification system is illustrated. The agent interface may be provided to a terminal or computer for an agent 10. The agent interface 110 allows the agent 10 to manage images received from different users 30.
[0034] The agent interface 110 may include an identifier 111 including a button or other user input to cause the product recognition server 100 to perform image processing techniques on the original image provided by the user 30 .
[0035] The agent interface 110 may include a connector 112 comprising a button or other user input to cause the product identification server 100 to send a link to the user 30. The link may be an address to initiate the communication session 101.
[0036] The agent interface 110 may include a client list 113 that includes a list of users 30 who have sent a link, have an active communication session 101, or have submitted an original image. The agent 10 may use the client list 113 to access analysis from image processing technology. The analysis may describe the model, SKU number, name, or other identification code or identifying characteristics of the product depicted in the original image.
[0037] Figure 3 An example terminal user interface 120 for a customer service and product identification system is illustrated. The user interface 120 may be provided by a mobile application on the end-user device 103. The terminal user interface 120 may include a user input 121, a navigation portion 122, and a results portion 123. Additional, different, or fewer components may be included.
[0038] The user input 121 may initiate operation of a camera application on the end-user device 103. The user input 121 causes the end-user device 103 to capture an original image. The user points the end-user device 103 at a product of interest and captures an image. The user input 121 may initiate operations including, for example, Figure 4 A dedicated camera application for the described template.
[0039] In addition to or as an alternative to image capture, guidance portion 122 can instruct the user where to find a label or include a mark to identify the product. The mark can include a quick read (QR) code, a barcode, or alphanumeric characters to identify the product. In some examples, the user directly observes the identity of the product from the label or code. In other examples, when applicable, the user captures an image of the label or code for subsequent analysis and identification of the product.
[0040] The results section 123 can display or otherwise provide the image analysis results at the product identification server 100 to the user at the end-user device 103. The results section 123 may include the name or model of the identified product. The results section 123 may also provide links or files related to documentation for the identified product, alternatives to the identified product, or manuals and literature for the identified product.
[0041] Figure 4An example product recognition camera interface 130 for a customer service and product recognition system is illustrated. Interface 130 may include a template 131 for matching a product 132 in an image. Template 131 may be selected based on a product classification, which may be input by a user or agent or determined through preliminary image analysis. The user may be instructed to move the end-user device 103 and the camera's view to align the product 132 with the template 131. In some examples, when the user observes alignment of the product 132 with the template 131, the user presses a capture button 133 to capture the image. In other examples, when the product 132 and template 131 become substantially aligned, the end-user device 103 may automatically trigger the capture of the image.
[0042] Figure 5 Illustrated is a summary interface 140 for an agent 10 or product identification server 100. The summary interface 140 includes product images provided by various users 30 and includes corresponding results for the product identification system.
[0043] Figure 6 An example learning model sequence 150 for a customer service and product recognition system is illustrated. The learning model sequence 150 is an example image processing technique that utilizes multiple learning models, such as neural networks, to analyze one or more images of a bathroom product. The learning model sequence 150 may include a first learning model 161 or a first neural network, a second learning model 162 or a second neural network, and a third learning model 163 or a third neural network. Each model may be associated with an application programming interface (API). The API for the first model 161 is an object detection API configured to identify products in an image. The API for the second model 162 is a raw image API configured to analyze raw images that include objects other than the object of interest. The API for the third model 163 is a cropped image API configured to analyze cropped images that closely surround the object of interest. The API for the stack model 164 combines the other three APIs.
[0044] The original image 151 is input to a learned model sequence 150, which provides an output 171 including at least one prediction value for identifying a bathroom product. Additional, different, or fewer components may be included.
[0045] The learned model sequence 150 receives an original image 151 captured by a user 30. The original image 151 depicts an unidentified bathroom product. In some examples, the learned model sequence 150 may also receive supplemental data from the user 30. The supplemental data may include a classification of the bathroom product. This classification describes the category or type of bathroom product, but does not identify a specific model or SKU of the bathroom product. Example classifications may include faucets, toilets, urinals, showerheads, sinks, bathtubs, or other types of bathroom products.
[0046] The learning model sequence 150 uses a first model 161 to analyze the original image 151. The first model 161 can be a neural network trained on images of bathroom products with known models. The ground truth image set (e.g., training images) can be provided by agents at the customer service center. The ground truth image set is images in which the products depicted have been skillfully identified by experienced agents 10. The product recognition server 100 can provide a verification interface 180 for identifying the training images, such as Figure 7 As shown, verification interface 180 provides an image provided by user 30 and accepts recognition input from agent 10. Verification interface 180 can display multiple images simultaneously. Multiple agents 10 can be provided with the same image for redundant recognition. The recognition input becomes the ground truth for training learning models 161, 162, and 163. When multiple agents provide ground truth information and recognize incoming images as they are received by product recognition server 100, the models can be updated in near real time.
[0047] In addition, multiple versions of the learning model 161, the learning model 162, and the learning model 163 can be developed (e.g., trained) for different product categories. Example categories include bathroom faucets, kitchen faucets, single-hole faucets, three-hole faucets, sinks, bathtubs, bathtub faucets, showerheads, or other examples. The product recognition server 100 can receive classification information of bathroom fixtures from the user 30 or the agent 10, and the product recognition server 100 can select one of the possible first model 161, the second model 162, and / or the third model 163 from a set of possible models based on the classification information.
[0048] The learning model sequence 150 provides the original image to multiple learning models in parallel (e.g., parallel path 173) or serially (e.g., serial path 172) based on the results of the first model 161. The results of the first model 161 can be evaluated based on the confidence score or predicted probability from the first model 161. When the results of the first model 161 have a high confidence value (e.g., above the confidence threshold), the product recognition server 100 determines that the object is detected and proceeds to the parallel path 173. High confidence can also be indicated by the object detection API (first model 161) returning a bounding box. The bounding box can be a set of coordinates (xmin, ymin, xmax, ymax) of a selected portion of the uploaded image that indicates the detected object.
[0049] When the results of the first model 161 have a low confidence value (e.g., below a confidence threshold), the product recognition server 100 determines that no object was found and proceeds to serial path 172. A low confidence score can also be indicated by the object detection API (first model 161) endpoint failing to detect any products or SKUs in the uploaded image, and the API returning "xmin:None" as a response.
[0050] In parallel path 173, product recognition server 100 can provide the raw image to second model 162. Second model 162 includes a raw image API that outputs another predicted value and predicted probability for the object in the image. Second model 162 provides a higher confidence threshold for the final prediction.
[0051] In parallel path 173, the product recognition server 100 crops the image using the bounding box coordinates from the original image. The cropped image 152 is provided to a third model 163 that has been trained on the cropped image. The third model 163 outputs a prediction value from the cropped image 152 and outputs a corresponding confidence value or prediction probability.
[0052] The prediction and probability outputs from the first model 161 (an object detection API configured to identify products in an image), the second model 162 (a raw image API configured to analyze a raw image that includes objects other than the object of interest), and the third model 163 (a cropped image API configured to analyze a cropped image) are provided to the fourth model or stack model 164. The API for the stack model 164 combines the prediction and probability outputs from the other three APIs or models and outputs a final prediction for output 171, which is displayed to the agent 10 and / or user 30. Additionally, output 171 may include a bounding box on the object and include a label for the product or SKU.
[0053] In serial path 172, when the first model 161 outputs a low prediction probability or confidence score, the product recognition server 100 forwards the raw image 151 to the second model 162. The second model 162 includes a raw image API that outputs another prediction value and prediction probability for the object in the image. The second model 162 provides a higher confidence threshold.
[0054] In addition, in response to a low prediction probability or confidence score, the product recognition server 100 generates a warning to the user (e.g., object not found) and prompts the user to crop the image. In serial path 172, the product recognition server 100 can prompt the user to crop the object in the image using the cropper 153 (cropping tool or cropping module) to crop (select a portion of the image) the object in the image. Thus, in this example, the user provides a bounding box to create a cropped image 152, which is provided to the third model 163.
[0055] The third model 163 (cropped image API) then provides the final prediction and predicted probability as output 171 to be displayed to the agent 10 and / or user 30. Additionally, the output 171 may include a bounding box on the object and include a label for the product or SKU.
[0056] In this way, two separate independent workflows are provided for analyzing images acquired by the user. The choice of workflow depends on the results of the first model 161 .
[0057] Information can be accessed based on output 171 and provided to user 30 or dealer 10. In one example, product identification server 100 accesses a parts database in response to the predicted value of the product, and product identification server 100 sends data from the parts database to user 30 based on the identified product. In response to output 171, user 30 can automatically order the part or be prompted with the option to order the part.
[0058] In one example, the product identification server 100 accesses a troubleshooting database in response to the predicted value of the bathroom product. Information from the troubleshooting database is provided to the agent 10 to help the user 30 troubleshoot the problem. In addition, data from the troubleshooting database is sent directly to the user 30.
[0059] In one example, the product identification server 100 accesses a substitute database in response to the predicted value of the product. The product identification server 100 can provide substitute information to the agent 10 or user 30 to replace the identified product. In response to output 171, the user 30 can be automatically provided with a substitute product or be prompted with the option of ordering a substitute product.
[0060] In one example, the product identification server 100 accesses a complementary product database in response to the predicted value of the product. The product identification server 100 may provide product information or a product model number to the agent 10 or user 30 to supplement the identified product. In response to output 171, the user 30 may be automatically provided with the complementary product or be prompted with the option of ordering the complementary product.
[0061] Figure 8 An example embodiment is shown in which multiple images of a product from multiple angles or perspectives are used in training. Product recognition server 100 can display a multi-view interface 190 for agent 10. Customer list 113 may include multiple images of the product discussed during communication session 101. The agent can then verify all of these images of individual parts or components of the product from multiple angles to train the overall set of product SKUs. The agent can then verify all images associated with a single recognized image, further strengthening the model.
[0062] Figure 9 An example embodiment is illustrated in which a partial view or exploded product is provided to the model. The partial view interface 210 can provide such a partial view or image of the exploded product or separated parts to the agent 10 via the product recognition system. The agent 10 can apply this viewpoint as additional ground truth to the model. This set of ground truth images can include the partial view or exploded product. Subsequent images with partial views or exploded products can be recognized by the model. In this way, expert recognition is accomplished on one image by the model, or the expert is trained on multiple other images of associated product views that would not normally be known or recognized by the model or expert unless such association was made in the design / invention of the system. For troubleshooting purposes, various angles of the product are shared due to the nature of the product being damaged and not shown in its original form.
[0063] Figure 10 The diagram illustrates an example embodiment in which new categories or classifications of products are generated. The agent 10 or the model can identify product types not currently in the model. A new product interface 220 is used to establish a training set for products known to be unrecognized by the model by allowing the agent to manually identify untrained categories to build sufficient training data to create the model's recognized categories. In the illustrated example, there is no training data for the shower door category. When the model identifies a predetermined number of customer needs, the agent 10 can be prompted to train new categories for future images.
[0064] Figure 11An example controller 301 for product identification and for customer assistance in response to product identification is illustrated. The product identification server 100 can implement the controller 301. The controller 301 can include a processor 300, a memory 352, and a communication interface 353 for interfacing with a device or with the Internet and / or other network 346. In addition to the communication interface 353, a sensor interface can be configured to receive data from a sensor (e.g., a proximity sensor).
[0065] The communication interface 353 is configured to receive an original image of a plumbing product (e.g., from the end-user device 103) and provide the original image to the processor 300. The memory 352 is configured to store a first model configured to analyze the original image of the plumbing product, a second model configured to analyze the original image of the plumbing product in combination with supplemental information about the plumbing product, and a third model configured to analyze a cropped image of the plumbing product.
[0066] The processor 300 is configured to perform analysis using the first model, the second model, and the third model such that when the first model indicates an object match for the bathroom product in the original image, the second model and the third model are executed consecutively, and when the first model lacks an object match for the bathroom product in the original image, the second model and the third model are executed in parallel.
[0067] Components of the control system may communicate using bus 348. The control system may be connected to a workstation or another external device (eg, a control panel) and / or a database for receiving user input, system properties, and any values described herein.
[0068] Optionally, the control system may include an input device 355 and / or sensing circuitry / sensors 356 that communicate with any of the sensors. As described above, the sensing circuitry receives sensor measurements from the sensors. The input device may include any user input such as a button, a touch screen, a keyboard, a microphone for voice input, a camera for gesture input, and / or another mechanism.
[0069] Optionally, the control system may include a drive unit 340 for receiving and reading a non-transitory computer medium 341 having instructions 342. Additional, different, or fewer components may be included. The processor 300 is configured to execute the instructions 342 stored in the memory 352, thereby executing the algorithms described herein. The display 350 may be a pointer or other screen output device. The display 350 may be combined with a user input device 355.
[0070] Figure 12 Pictured Figure 11The actions of the flowchart may be performed by the controller 301 implemented by the product identification server 100. Additional, different, or fewer actions may be included.
[0071] In action S101, the controller 301 (e.g., the processor 300) receives an original image captured by a user, the original image depicting a bathroom product. The original image can be captured by the user using a camera or other type of image sensor (e.g., the sensor 356). The user can initiate contact with the controller 301 via phone, email, or other forms of communication. The original image captured by the user can be sent to the controller 301 via a second form of communication, which can be authenticated using a user account, warranty certificate, or other authentication technology.
[0072] In action S103, the controller 301 (e.g., the processor 300) provides a first model to analyze the original image. The first model can be a first neural network. The first model is configured to identify or otherwise match bathroom products in the original image. The first model can be matched with bathroom products from a previously stored set of faucets, sinks, showerheads, urinals, toilets, or other devices. In action S105, the controller 301 (e.g., the processor 300) determines whether the results of the first model include a match. When no match is determined, the user is prompted to crop the image, for example, via the display 350 and / or input device 355. The user can crop the image by highlighting the bathroom products in the image and effectively reducing the size of the image.
[0073] In action S107, when the first model indicates an object match for the bathroom product in the original image, the controller 301 (e.g., the processor 300) provides a second model for the original image and a third model for a cropped version of the original image in parallel. The second model may be a second neural network, and the third model may be a third neural network.
[0074] In action S109 , when the first model lacks an object match for the pipe product in the original image, the controller 301 (eg, processor 300 ) serially provides a second model for the original image and a third model for a cropped version of the original image.
[0075] In act S111 , the controller 301 (eg, the processor 300 ) outputs a predicted value for the bathroom product in response to the first model, the second model, and the third model.
[0076] The processor 300 may be a general-purpose or special-purpose processor, an application-specific integrated circuit (ASIC), one or more programmable logic controllers (PLCs), one or more field-programmable gate arrays (FPGAs), a set of processing components, or other suitable processing components. The processor 300 is configured to execute computer code or instructions stored in the memory 352 or received from other computer-readable media (e.g., embedded flash memory, local hard disk storage, local ROM, network storage, remote server, etc.). The processor 300 may be a single device or a combination of devices, such as those associated with a network, distributed processing, or cloud computing.
[0077] The memory 352 may include one or more devices (e.g., memory cells, memory devices, storage devices, etc.) to store data and / or computer code for completing and / or facilitating the various processes described in this disclosure. The memory 352 may include random access memory (RAM), read-only memory (ROM), hard drive storage, temporary storage, non-volatile memory, flash memory, optical storage, or any other suitable memory for storing software objects and / or computer instructions. The memory 352 may include a database component, an object code component, a script component, or any other type of information structure for supporting the various activities and information structures described in this disclosure. The memory 352 may be communicatively connected to the processor 300 via a processing circuit, and the memory 352 may include computer code for executing (e.g., executed by the processor 300) one or more processes described herein. For example, the memory 298 may include graphics, web pages, HTML files, XML files, script code, shower configuration files, or other resources that are used to generate a graphical user interface for display and / or to interpret user interface input to make command, control, or communication decisions.
[0078] In addition to the inlet port and the outlet port, the communication interface 353 can also include any operable connection. The operable connection can be a connection in which signals, physical communications and / or logical communications can be sent and / or received. The operable connection can include a physical interface, an electrical interface and / or a data interface. The communication interface 353 can be connected to a network. The network can include a wired network (e.g., Ethernet), a wireless network, or a combination thereof. The wireless network can be a cellular telephone network, 802.11, 802.16, 802.20, WiMax network, Bluetooth pairing of the device, or a Bluetooth mesh network. In addition, the network can be a public network such as the Internet, a private network such as an intranet, or a combination thereof, and the network can utilize various network protocols that are now available or developed later, including but not limited to network protocols based on TCP / IP.
[0079] Although the computer-readable medium (e.g., memory 352) is shown as a single medium, the term "computer-readable medium" includes a single medium or multiple media such as a centralized or distributed database and / or associated caches and servers that store one or more sets of instructions. The term "computer-readable medium" should also include any medium that can store, encode, or carry a set of instructions for execution by a processor or cause a computer system to perform any one or more of the methods or operations disclosed herein.
[0080] In a specific non-limiting exemplary embodiment, the computer-readable medium may include a solid-state memory such as a memory card or other package containing one or more non-volatile read-only memories. In addition, the computer-readable medium may be a random access memory or other volatile rewritable memory. Additionally, the computer-readable medium may include magneto-optical or optical media such as a disk or tape or other storage device to capture carrier signals such as signals transmitted through a transmission medium. Digital file attachments to e-mails or other self-contained information archives or archives may be considered as distribution media as tangible storage media. Therefore, the present disclosure is considered to include any one or more of computer-readable media or distribution media, and other equivalents and successor media in which data or instructions may be stored. Computer-readable media may be non-transitory and include all tangible computer-readable media.
[0081] In alternative embodiments, dedicated hardware implementations such as application specific integrated circuits, programmable logic arrays, and other hardware devices can be constructed to implement one or more of the methods described herein. Applications that may include the devices and systems of various embodiments may broadly include various electronic and computer systems. One or more embodiments described herein may implement functionality using two or more specific interconnected hardware modules or devices with associated control and data signals that may be transmitted between and through the modules or as part of an application specific integrated circuit. Therefore, the present system comprises software, firmware, and hardware implementations.
[0082] The illustrations of the various embodiments described herein are intended to provide an overall understanding of the structures of the various embodiments. These illustrations are not intended to be used as a complete description of all elements and features of the devices and systems utilizing the structures or methods described herein. Many other embodiments may be apparent to those skilled in the art upon review of this disclosure. Other embodiments may be utilized and other embodiments may be derived from this disclosure so that structural and logical replacements and changes may be made without departing from the scope of this disclosure. Additionally, these illustrations are representative only and may not be drawn to scale. Certain proportions in the illustrations may be exaggerated, while other proportions may be minimized. Therefore, this disclosure and the accompanying drawings should be considered to be illustrative and not restrictive.
[0083] Although this specification contains many details, these details should not be interpreted as limitations on the scope of the present disclosure or the scope of what may be claimed, but rather as descriptions of features peculiar to particular embodiments of the present disclosure. Certain features described in this specification in the context of separate embodiments may also be implemented in combination in a single embodiment. On the contrary, the various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any suitable sub-combination. In addition, although features may be described above as working in certain combinations, and even initially claimed as such, one or more features from the claimed combination may be removed from the combination in some cases, and the claimed combination may be directed to a sub-combination or a variation of the sub-combination.
[0084] One or more embodiments of the present disclosure may be referred to herein, individually and / or collectively, by the term "invention," which is merely for convenience and is not intended to actively limit the scope of this application to any particular invention or inventive concept. In addition, although specific embodiments have been illustrated and described herein, it should be understood that any subsequent arrangements designed to achieve the same or similar purposes may replace the specific embodiments shown. The present disclosure is intended to cover any and all subsequent adaptations or variations of the various embodiments. Combinations of the above embodiments and other embodiments not specifically described herein will be apparent to those skilled in the art upon review of this specification.
[0085] The foregoing detailed description is intended to be illustrative rather than restrictive, and it should be understood that the following claims, including all equivalents, are intended to define the scope of the disclosure. The claims should not be construed as limited to the described order or elements unless stated to that effect. Therefore, all embodiments that come within the scope and spirit of the appended claims and their equivalents are claimed as the present disclosure.
Claims
1. A method for identifying bathroom products, the method include: Receiving an original image captured by a user, wherein the original image depicts the bathroom product; providing a first model to analyze the original image; When the first model indicates an object match for the bathroom product in the original image, providing in parallel a second model for the original image and a third model for a cropped version of the original image; When the first model lacks the object match for the bathroom product in the original image, continuously providing the second model for the original image and the third model for the cropped version of the original image; and A predicted value of the bathroom product is output in response to the first model, the second model, and the third model.
2. The method according to claim 1, further comprising: include: When the first model indicates the object match for the bathroom product in the original image, the original image is cropped in response to the object match.
3. The method according to claim 1, further comprising: include: When the first model lacks the object match for the bathroom product in the original image, the user is prompted to manually crop the original image.
4. The method according to claim 1, further comprising: include: establishing a communication session with the user; as well as A link for capturing the original image is sent to the user via the communication session.
5. The method according to claim 4, in, The link includes identification information of the user.
6. The method according to claim 4, further comprising: include: Classification information of the bathroom product is received from the user, wherein the first model, the second model, or the third model is based in part on the classification information.
7. The method according to claim 1, in, When the first model lacks the object match for the bathroom product in the original image, providing the output of the first model as input to the second model and providing the output of the second model as input to the third model.
8. The method according to claim 1, further comprising: include: When the first model indicates the object match for the sanitary product in the original image, a second model for the original image and a third model for a cropped version of the original image are provided in parallel.
9. The method according to claim 1, further comprising: include: accessing a parts database in response to the predicted value of the bathroom product; as well as Data from the parts database is sent to the user.
10. The method according to claim 1, further comprising: include: accessing a troubleshooting database in response to the predicted value of the plumbing product; as well as Data from the troubleshooting database is sent to the user.
11. The method according to claim 1, further comprising: include: accessing a replacement database in response to the predicted value of the plumbing product; as well as Data from the alternative database is sent to the user.
12. The method according to claim 1, further comprising: include: accessing a supplemental product database in response to the predicted value of the plumbing product; as well as Data from the supplemental product database is sent to the user.
13. A device for identifying bathroom products, the device include: a communication interface configured to receive an original image of the bathroom product; a first model configured to analyze the raw image of the sanitary product; a second model configured to analyze the original image of the sanitary product in combination with supplemental information of the sanitary product; a third model configured to analyze a cropped image of the bathroom product; as well as A controller configured to perform analysis using the first model, the second model, and the third model, wherein the second model and the third model are executed consecutively when the first model indicates an object match for the sanitary product in the original image, and the second model and the third model are executed in parallel when the first model lacks the object match for the sanitary product in the original image.
14. The device according to claim 13, in, A predicted value of the bathroom product is output in response to the first model, the second model, and the third model.
15. The device according to claim 13, in, The controller generates a request for acquiring the original image of the bathroom product.
16. The device according to claim 13, in, The bathroom products include sinks, faucets, shower heads, toilets or urinals.
17. The device according to claim 13, in, The first model includes a first neural network, the second model includes a second neural network, and the third model includes a third neural network.
18. The device according to claim 13, in, The communication interface is configured to provide a first communication session between an end-user device and a customer service center device and to provide a second communication session between the end-user device and the customer service center device.
19. The device according to claim 18, in, The first communication session comprises a voice or video call and the second communication session comprises a file transfer.
20. A non-volatile computer readable medium comprising instructions which, when executed, are configured to perform a method, the method include: Receiving an original image captured by a user, wherein the original image depicts a bathroom product; providing a first model to analyze the original image; When the first model indicates an object match for the bathroom product in the original image, providing in parallel a second model for the original image and a third model for a cropped version of the original image; When the first model lacks the object match for the bathroom product in the original image, continuously providing the second model for the original image and the third model for the cropped version of the original image; and A predicted value of the bathroom product is output in response to the first model, the second model, and the third model.
