Method and apparatus for searching for product, and recording medium

Abstract

The present disclosure relates to: a method by which an electronic device searches for a product; an apparatus; and a recording medium. The method by which an electronic device searches for a product, according to one embodiment of the present disclosure, may comprise the steps of: receiving a first image corresponding to a product search query from a user terminal; in response to receiving the first image, searching for a product corresponding to the first image on the basis of an image set, the image set including a product description image and / or a product review image; and transmitting the product search result to the user terminal.

Description

Methods, devices and recording media for searching products

[0001] The present invention relates to a technology for searching for products. More specifically, it relates to a technology for searching for products based on images in an e-commerce service.

[0002] Advances in electronic communication technology and online commerce platforms have made it increasingly convenient for users to search for and purchase a variety of products online via their electronic devices. This growth in online commerce has coincided with the development of distribution networks, resulting in online purchases becoming comparable in terms of time and availability to offline purchases. Furthermore, more and more users are willing to order online if the same product is cheaper online than offline. Consequently, even while shopping offline, users are increasingly motivated to compare prices with online alternatives and make a purchase.

[0003] As the incentives to compare prices of offline and online products have increased, the need to accurately and quickly search for a variety of products online has arisen.

[0004] Techniques for searching products using images captured by a user's terminal's camera have been proposed, but their accuracy is limited. Barcode and QR code searches have also been attempted to improve accuracy, but these methods all have limitations due to the requirement to recognize a barcode or QR code for search purposes.

[0005] In addition, since conventional image-based search technology provided search results in real time based on images captured by users, there were limitations on the algorithms and data sets that could be used for search, and thus, there was a problem in that the accuracy of product search was not high.

[0006] Therefore, image search technology is required that can provide search results asynchronously rather than in real time, with high search accuracy and the products that users are looking for.

[0007] A technical problem to be solved through one embodiment of the present disclosure is to provide highly accurate search results through image-based search.

[0008] Another technical challenge to be solved through one embodiment of the present disclosure is to provide product search results asynchronously rather than in real time, thereby ensuring sufficient time to provide highly accurate product search results.

[0009] Another technical challenge to be solved by one embodiment of the present disclosure is to enable a user to search multiple products at once by taking a single image of each product without having to take a single image of each product.

[0010] The technical problems of the present disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art of the present disclosure from the description below.

[0011] A product search method performed by an electronic device according to one embodiment of the present disclosure may include: receiving a first image corresponding to a product search query from a user terminal; searching for a product corresponding to the first image based on an image set in response to receiving the first image, wherein the image set includes at least one image from among a product description image and a product review image; and transmitting a result of the product search to the user terminal.

[0012] In one embodiment, the first image may include at least one of an image acquired in real time through an input unit of the user terminal and an image previously stored in a storage unit of the user terminal.

[0013] In one embodiment, the product review image may be characterized in that it is updated in real time or periodically by the electronic device.

[0014] In one embodiment, the method may further include the steps of: receiving a second image corresponding to a product search query from a user terminal; generating a search item corresponding to the first image and a search item corresponding to the second image, respectively; and transmitting a search result to the user terminal in response to a determination that there is a search result of a search item corresponding to the first image or a search item corresponding to the second image.

[0015] In one embodiment, the method may further include: performing an image quality check of the received first image; and, if the result of the image quality check is determined to be failed, providing an image quality guide to the user terminal.

[0016] In one embodiment, the step of providing the image quality guide may further include the step of providing at least one guideline for image capturing to the user terminal, wherein the at least one guideline is at least one of geometric shapes representing a capturing range for capturing at least one of a product shape, a product barcode, and product information.

[0017] In one embodiment, the first image is an image pre-stored in the user terminal, and the electronic device is granted access to the pre-stored image, and when it is determined that the access is granted, the electronic device searches for a product corresponding to the first image without receiving the first image.

[0018] In one embodiment, the result of the product search may be transmitted as a push alarm.

[0019] In one embodiment, the method may further include the step of determining whether the first image includes images of a plurality of products; and in response to determining that the first image includes images of the plurality of products, the step of generating a search item corresponding to each image of the plurality of products.

[0020] In one embodiment, the step of transmitting the product search result to the user terminal may include the step of determining the priority of the product in the search result based on the product review rate; and the step of transmitting the product search result together with the priority to the user terminal.

[0021] In one embodiment, the step of receiving the first image may further include the step of applying post-processing imaging processing to the first image in response to receiving the first image.

[0022] In one embodiment, the post-processing imaging processing may include cropping or blurring based on the product included in the first image.

[0023] A product search method performed by a user terminal according to one embodiment of the present disclosure may include: a step of acquiring a first image corresponding to a product search query and sending the image to an electronic device; and a step of receiving a search result of a product corresponding to the first image based on an image set in response to receiving the first image from the electronic device, wherein the image set includes at least one image from among a product description image and a product review image.

[0024] In one embodiment, the first image may include at least one of an image acquired in real time through an input unit of the user terminal and an image previously stored in a storage unit of the user terminal.

[0025] An electronic device according to one embodiment of the present disclosure comprises one or more processors, one or more memories storing instructions executed by the one or more processors, and when the instructions are executed by the one or more processors, the one or more processors may be configured to execute a method according to any one of claims 1 to 14.

[0026] In a non-transitory computer-readable recording medium having recorded thereon instructions that, when executed by one or more processors according to one embodiment of the present disclosure, cause the one or more processors to perform an operation, the instructions may be configured to cause the one or more processors to perform a method according to any one of claims 1 to 14.

[0027] According to the present disclosure, by providing a highly accurate image-based search technology, it is possible to increase user convenience while inducing more purchases from users.

[0028] According to the present disclosure, product search results are provided asynchronously rather than in real time, thereby securing sufficient time to enable highly accurate product searches.

[0029] According to the present disclosure, a user can capture a single image of multiple products and search them at once, without having to capture an image of each product individually.

[0030] According to the present disclosure, users can intuitively search for desired products and obtain search results with greater accuracy. This increases the efficiency of product searches, enhancing the user experience and increasing the user usage rate of e-commerce services.

[0031] The effects according to the technical idea of ​​the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the specification.

[0032] FIG. 1 illustrates an environment in which a user terminal and an electronic device according to one embodiment of the present disclosure can be applied.

[0033] FIG. 2 is a block diagram of an electronic device according to one embodiment of the present disclosure.

[0034] Figure 3 is a flowchart illustrating a method according to one embodiment of the present disclosure.

[0035] FIG. 4 is a drawing showing a product description image according to one embodiment of the present disclosure.

[0036] FIG. 5 is a diagram illustrating a product review image and a product review user interface provided on a user terminal according to one embodiment of the present disclosure.

[0037] FIG. 6 is a diagram showing search items and search results corresponding to a first image and a second image according to one embodiment of the present disclosure.

[0038] FIG. 7 is a flowchart generally illustrating detailed steps of a method according to one embodiment of the present disclosure.

[0039] FIGS. 8A, 8B, 8C and 8D are flowcharts detailing steps of a method according to one embodiment of the present disclosure.

[0040] Figure 8a illustrates a step (S710) of receiving a first image from a user terminal (120).

[0041] Figure 8b illustrates a step (S720) of providing an image quality check.

[0042] Figure 8c illustrates a step (S730) of searching for a product corresponding to a received image.

[0043] FIG. 8d illustrates terminating image search by providing search results to search items (S740).

[0044] The various embodiments described in this disclosure are exemplified for the purpose of clearly explaining the technical concept of this disclosure and are not intended to be limited to specific embodiments. The technical concept of this disclosure includes various modifications, equivalents, alternatives, and embodiments selectively combined from all or part of the embodiments described in this disclosure. Furthermore, the scope of the technical concept of this disclosure is not limited to the various embodiments presented below or the specific descriptions thereof.

[0045] Terms used in this disclosure, including technical or scientific terms, unless otherwise defined, may have the meaning commonly understood by a person of ordinary skill in the art to which this disclosure belongs.

[0046] The expressions "includes," "may include," "comprises," "may have," "have," and "may have" used in this disclosure indicate the presence of a target feature (e.g., a function, operation, or component), but do not exclude the presence of other additional features. In other words, such expressions should be understood as open-ended terms that imply the possibility of including other embodiments.

[0047] The singular expressions used in this disclosure may include the plural meaning unless the context clearly indicates otherwise, and the same applies to the singular expressions set forth in the claims.

[0048] The expressions "first," "second," or "first", "second", etc. used in this document, unless the context indicates otherwise, are used to refer to multiple similar objects and to distinguish one object from another, and do not limit the order or importance among the objects.

[0049] As used herein, the expressions "A, B, and C", "A, B, or C", "A, B, and / or C", or "at least one of A, B, and C", "at least one of A, B, or C", "at least one of A, B, and / or C", etc., may refer to each of the listed items or to all possible combinations of the listed items. For example, "at least one of A or B" may refer to (1) at least one A, (2) at least one B, (3) at least one A and at least one B.

[0050] The expression "based on" as used in this disclosure is used to describe one or more factors that influence a decision, act of judgment, or action described in a phrase or sentence containing this expression, and this expression does not exclude additional factors that influence the decision, act of judgment, or action.

[0051] As used herein, the expression that a component (e.g., a first component) is “connected” or “connected” to another component (e.g., a second component) may mean that the component is directly connected or connected to the other component, as well as connected or connected via a new other component (e.g., a third component).

[0052] The expression "configured to" used in the present disclosure may have the meanings of "set to", "having the ability to", "modified to", "made to", "capable of", etc., depending on the context. This expression is not limited to the meaning of "specifically designed in hardware", and for example, a processor configured to perform a specific operation may mean a general purpose processor that can perform the specific operation by executing software, or a special purpose computer that is structured through programming to perform the specific operation.

[0053] Hereinafter, various embodiments of the present disclosure will be described with reference to the attached drawings. In the attached drawings and the description of the drawings, identical or substantially equivalent components may be assigned the same reference numerals. Furthermore, in the description of various embodiments below, duplicate descriptions of identical or corresponding components may be omitted, but this does not mean that such components are not included in the embodiments.

[0054] Figure 1 illustrates an environment in which an electronic device according to one embodiment of the present disclosure can be applied. An electronic device (110) and a user terminal (120) are connected via a network (130) and can communicate with each other.

[0055] The electronic device (110) may be a server device that provides an e-commerce service. That is, the electronic device (110) may be a server device operated under the management of an e-commerce service operator. The electronic device (110) may be a server device that manages a platform that provides an e-commerce service. Users can subscribe to the e-commerce service and purchase products through the e-commerce service.

[0056] The electronic device (110) can manage information about products provided by an e-commerce service. An e-commerce service may provide users with a service that allows them to purchase products such as goods, food, or products. Alternatively, the e-commerce service provided by the electronic device (110) may include a service that mediates the sale of products by some users to other users. Accordingly, the product-related information managed by the electronic device (110) may include, for example, product identification information, product information, product description images provided with the product information, product reviews, product review images attached to product reviews, product name, product price, product capacity, product size, product weight, product attributes, category, or unit price information. Managing product-related information can collectively refer to all actions that control and process the information. For example, managing product-related information may include collecting, storing, updating, or modifying the information. In one embodiment, product-related information may be updated in real time or periodically by the electronic device (110).

[0057] An electronic device (110) can manage information about customers using e-commerce services. The customer information managed by the electronic device (110) may include, for example, the customer's name, age, address, payment method, purchase history, or contact information. Managing customer information can encompass a series of actions involving controlling and processing that information. For example, managing customer information can include storing, updating, or modifying that information.

[0058] The electronic device (110) can perform a search related to an e-commerce service based on a user's request. In one embodiment, the electronic device (110) can perform a product search based on a request from a user using the e-commerce service (e.g., an image captured by the user). Additionally or alternatively, in one embodiment, the electronic device (110) can perform a product search based on information about the customer. Specifically, the electronic device (110) can perform a product search based on purchase history information.

[0059] The electronic device (110) can manage various information to be provided to users using e-commerce services. Based on requests from an administrator managing the e-commerce service or a user using the e-commerce service, the electronic device (110) can manage various data necessary to provide e-commerce services. For example, the electronic device (110) can create, modify, store, or provide to users data related to various pages associated with the e-commerce service, such as product list pages, product detail description pages, search results pages, product sales pages, event pages, and landing pages requested by users.

[0060] In addition, the electronic device (110) can manage transactions related to e-commerce services provided to users, i.e., transactions related to online products (e.g., goods or services). That is, the electronic device (110) can manage a series of processes, such as enabling users to search for, select, purchase, pay for, and request delivery of products. As such processes, the electronic device (110) can provide customers with information regarding products and delivery, and receive purchase, payment, and delivery requests from users, so that users can search for, select, purchase, pay for, and request delivery of products. In addition, it should be noted that even if an operation that the electronic device (110) can perform is not mentioned in the present disclosure, if it is a general operation that a server device providing a known e-commerce service can perform, applying the technical ideas of the present disclosure with reference to such an operation is not excluded from the scope of the present disclosure.

[0061] The electronic device (110) described above may be implemented as one or more computing devices. For example, all functions of the electronic device (110) may be implemented in a single computing device. For another example, a first function of the electronic device (110) may be implemented in a first computing device, and a second function may be implemented in a second computing device. As another example, multiple computing devices may be used, each of which implements all or specific functions of the electronic device (110). The computing devices described above may be, but are not limited to, a desktop computer, a laptop computer, an application server, a proxy server, or a cloud server, and any type of device equipped with computing functions may be a computing device.

[0062] The user terminal (120) may be a device for a user to use an e-commerce service. The user terminal (120) may be implemented as a terminal capable of transmitting and receiving various information with the electronic device (110) via a network. For example, the user terminal (120) may be one of a computer, a laptop, a portable communication terminal (e.g., a smartphone), a portable multimedia device, a wearable device, or an HMD. However, the type of the user terminal (120) is not limited thereto.

[0063] The user terminal (120) may be any device including an input unit (e.g., a camera) for acquiring an image and an output unit capable of outputting information to the user, and a communication unit capable of communicating with an electronic device (110) or other devices via a network (130). In addition, the user terminal (120) may include a storage unit capable of storing acquired images.

[0064] The user terminal (120) can acquire images in real time through the input unit. Furthermore, the user terminal (120) can store the acquired images in the storage unit. For example, the images acquired in real time through the input unit can be stored in the storage unit. The user terminal (120) can access images previously stored in the storage unit.

[0065] The user terminal (120) can transmit input to the electronic device (110) and output information received from the electronic device (110). For example, the user terminal (120) can transmit a product search query to the electronic device (110). Specifically, the user terminal (120) can acquire an image in real time through the input unit of the user terminal and / or include a pre-stored image in the product search query and transmit it to the electronic device (110). The input unit of the user terminal (120) can receive various forms of input, such as a camera, a click using a mouse, a touch using a touch pad or a touch screen, voice recognition, and other electronic inputs.

[0066] The user terminal (120) can receive various pages from the electronic device (110) and output the various pages received. The user terminal (120) can provide the search results received from the electronic device (110) to the user. The user terminal (120) can receive search results with priorities from the electronic device (110) and output the results to the user in the order of the priorities.

[0067] The network (130) may serve to connect the electronic device (110) and the user terminal (120) or other external devices. For example, the network may provide a connection path so that the user terminal (120) or other external devices can be connected to the electronic device (110) and transmit and receive packet data with the electronic device (110). The network may be implemented as any type of wired or wireless network, such as a local area network (LAN), a wide area network (WAN), a mobile radio communication network, or Wibro (Wireless Broadband Internet).

[0068] In one embodiment, the electronic device (110) and the user terminal (120) may operate as a single device. The user terminal (120) may be included in the electronic device (110) as a component of all or part of the electronic device (110). In this case, for example, various types of information exchanged between the electronic device (110) and the user terminal (120) via a network may be various types of information exchanged between components within a single device.

[0069] FIG. 2 is a block diagram of an electronic device according to one embodiment of the present disclosure. The electronic device (200) can process information regarding an e-commerce service. In one embodiment, the electronic device (200) may include one or more processors (210), one or more memories (220), and a communication interface (230) as components. In one embodiment, at least one of the components of the electronic device (200) may be omitted, or another component may be added to the electronic device (200). In one embodiment, additionally or alternatively, some of the components may be implemented in an integrated manner or implemented as a single or multiple entities. In the present disclosure, one or more processors (210) may be referred to as a processor (210). The expression “processor (210)” may mean a set of one or more processors, unless the context clearly indicates otherwise. In the present disclosure, one or more memories (220) may be referred to as a memory (220). The expression "memory (220)" may mean a set of one or more memories, unless the context clearly indicates otherwise. In one embodiment, at least some of the components inside / outside the electronic device (200) may be connected to each other via a bus, a General Purpose Input / Output (GPIO), a Serial Peripheral Interface (SPI), or a Mobile Industry Processor Interface (MIPI), and may exchange information (data, signals, etc.).

[0070] The processor (210) may control at least one component of an electronic device (200) connected to the processor (210) by executing software (e.g., commands, programs, etc.). In addition, the processor (210) may perform various operations related to the present disclosure, such as calculations, processing, data generation, and processing. In addition, the processor (210) may load data from the memory (220) or store data in the memory (220). In one embodiment, the processor (210) may control the communication interface (230) to request various types of information from the user terminal (120) and receive various types of information from the user terminal (120).

[0071] The processor (210) may generate a page related to an e-commerce service and provide information about the generated page to the user's terminal. For example, the page provided by the processor (210) may include a main page that is first displayed when a user accesses the e-commerce service. The main page may include at least a portion of an area that displays information about one or more products provided to the user. The main page may also include at least a portion of an area for entering search terms corresponding to a search query and / or an icon for initiating a search. In one embodiment, the page provided by the processor (210) to the user may include a product search results page. The product search results page may include multiple product search items. Each search item may correspond to a product search query.

[0072] The memory (220) can store various information (data). The information stored in the memory (220) is information acquired, processed, or used by at least one component of the electronic device (200), and may include software (e.g., commands, programs, etc.). The memory (220) may include volatile and / or non-volatile memory. In the present disclosure, the commands or programs are software stored in the memory (220), and may include an operating system for controlling the resources of the electronic device (200), an application, and / or middleware for providing various functions to the application so that the application can utilize the resources of the electronic device (200). In one embodiment, the memory (220) may store commands that, when executed by the processor (210), cause the processor (210) to perform operations. The memory (220) may store at least a portion of information received from a database via the communication interface (230) and / or information transmitted to the database via the communication interface (230). Specifically, the memory (220) can store information regarding e-commerce and commands executed by the processor (210).

[0073] The communication interface (communication interface, 330) can perform wireless or wired communication between the electronic device (200) and a database or other external electronic device. For example, the communication interface (230) can perform wireless communication according to a method such as eMBB (enhanced Mobile Broadband), URLLC (Ultra Reliable Low-Latency Communications), MMTC (Massive Machine Type Communications), LTE (Long-Term Evolution), LTE-A (LTE Advance), NR (New Radio), UMTS (Universal Mobile Telecommunications System), GSM (Global System for Mobile communications), CDMA (Code Division Multiple Access), WCDMA (Wideband CDMA), WiBro (Wireless Broadband), WiFi (Wireless Fidelity), Bluetooth (Bluetooth), NFC (Near Field Communication), GPS (Global Positioning System), or GNSS (Global Navigation Satellite System). For example, the communication interface (230) may perform wired communication according to a method such as Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI), Recommended Standard-232 (RS-232), or Plain Old Telephone Service (POTS). In one embodiment, the electronic device (200) may be implemented by being integrated with another device. In this case, the communication interface (230) may function as a connection circuit or interface connecting the electronic device (200) and the other device.

[0074] Hereinafter, the operations described as being performed by the electronic device in FIG. 3 and FIG. 8a to FIG. 8d can be understood as being performed by the processor (210) of the electronic device (200) described in FIG. 2.

[0075] Figure 3 is a flowchart illustrating a method according to one embodiment of the present disclosure.

[0076] The method disclosed in FIG. 3 can be understood as a series of operations that provide image-based product search results to an e-commerce platform or users utilizing e-commerce. The operations illustrated in FIG. 3 are described in detail below.

[0077] The electronic device (110) can receive a first image corresponding to a product search query (S310).

[0078] In one embodiment, the electronic device (110) may receive a first image corresponding to a product search query from a user terminal (120). In the present disclosure, a product search query may collectively refer to a user accessing an e-commerce service and extracting products and / or related products that the user wishes to purchase from a product database within or external to the electronic device.

[0079] In one embodiment, the first image may include at least one of an image acquired in real time through the input unit of the user terminal and an image previously stored in the storage unit of the user terminal. Here, the first image may include an image of a product that the user wishes to search for. For example, the first image may be an image that includes product information such as the side, front, back, rear, product name, barcode, production date, contents, seller, brand, and purpose of the product that the user wishes to search for. The user terminal (120) may transmit a product search query including the first image to the electronic device (110) via the network (130).

[0080] In one embodiment, the user terminal (120) may receive a decision from the user regarding whether to use an image previously stored in the storage unit. In response to a decision not to use a previously stored image, the user terminal (120) may acquire an image in real time through an input unit (e.g., a camera) of the user terminal (120). Additionally or alternatively, in response to a decision to use a previously stored image, the user terminal (120) may acquire an image previously stored in the storage unit of the user terminal (120).

[0081] Additionally or alternatively, the user terminal (120) may receive a text search term from the user and transmit it as a query. The query is intended to inquire about the same or similar product to the electronic device (110), and may take the form of a URL, image, text, or the like.

[0082] In response to receiving the first image, the electronic device (110) can search for a product corresponding to the first image based on the image set (S320).

[0083] In one embodiment, the image set may include at least one of a product description image and a product review image. The product description image and the product review image may be the images illustrated in FIGS. 4 and 5 , respectively. In one embodiment, the product description image and / or the product review image may be updated in real time or periodically by the electronic device (110).

[0084] Below, the product description image and product review image are described in more detail through FIGS. 4 and 5.

[0085] FIG. 4 is a diagram illustrating a product description image according to one embodiment of the present disclosure. In the present disclosure, a product description image (400) is information regarding a product managed by an electronic device (110), and may collectively refer to an image provided to a user terminal via a product description page along with product information by the electronic device (110).

[0086] For example, the product description image (400) may be an image of the product included in the product description or provided together with the product description to enhance product identification. Specific image processing may be applied to the product description image. Image processing may be a series of graphic processing techniques for improving the quality of the product description image. Specifically, the image processing may be graphic processing techniques such as noise reduction, sharpening, contrast adjustment, brightness correction, resolution enhancement, geometric corrections, color balance adjustment, and brightness and color adjustments. The product description image (400) to which such specific image processing is applied may vary greatly depending on the capturing method and may be distinguished from an image directly captured by the user.

[0087] FIG. 5 is a diagram illustrating a product review image and a user interface for a user to write a product review after purchasing a product according to one embodiment of the present disclosure. The product review user interface (500) may be provided by an electronic device (110) and refers to an interface for an e-commerce service user to input their experience of purchasing and using a product. The product review user interface (500) may include at least a portion of an area for receiving a product review rate (510), which is a rating indicating the user's satisfaction with the purchased product. The product review user interface (500) may include at least a portion of an area for receiving a product review (520) that describes in writing the user's satisfaction or evaluation of the purchased product.

[0088] In addition, the product review user interface (500) may include at least a portion of an area for receiving product review images (530, 540, 550, 560). A product review image refers to one or more images that are taken by a user and attached to a user review to visually show the appearance, function, usage, etc. of a product. Specifically, a product review image may be any image related to a product, such as the front (530), back (540), side view, contents (550) including accessories, production date (560), barcode, package, and packaging. In addition, these images (530, 540, 550, 560) may be collectively referred to as product review images. For example, in the case of a clothing product, a product review image may be an image showing the product being worn, the product from various angles, or an image showing the texture of the material of the product. Additionally, for example, in the case of electronic products, the images may show the product's display screen, which may represent the product's appearance and / or interior, the running of apps, etc., how it operates, accessories and inclusions provided with the product (e.g., chargers, cases, etc.), etc.

[0089] In one embodiment, the product review user interface (500) may provide guidelines for the types of product review images (530, 540, 550, 560) in at least a portion of an area where product review images (530, 540, 550, 560) can be input. For example, guidelines with text such as "front shot," "back shot," "contents," and "production date" may be provided in a portion of an area where product review images (530, 540, 550, 560) can be input.

[0090] Those skilled in the art will understand that the more diverse the types of images input as product review images (530, 540, 550, 560), the more diverse the image set becomes, and the more information can be provided to the electronic device (110). Those skilled in the art will also understand that the more diverse the image set, the more accurate the search. According to the present disclosure, by providing a technology that accurately searches for a product desired by a user while performing an image-based search, user convenience can be enhanced while also encouraging more purchases from the user.

[0091] Product review rates (510), product reviews (520), product review images (530, 540, 550, 560), etc. entered by these users can be managed by the electronic device (110). Product review images (530, 540, 550, 560) can be managed by the electronic device (110) in association with products.

[0092] The electronic device (110) performs various operations to search for products. These search operations are described in detail later with reference to FIG. 8c.

[0093] Referring again to the operations illustrated in FIG. 3, the electronic device (110) can transmit the results of a product search to the user terminal (120) (S330). The electronic device (110) can transmit the results of searching for a product corresponding to the first image based on the image set to the user terminal (120). In the present disclosure, the search results may include products having images that are highly aligned with the first image.

[0094] FIG. 6 is a diagram illustrating an example of a user interface that provides product search results according to one embodiment of the present disclosure, showing a plurality of search items (600) and a search result (660) for one search item.

[0095] An information search method according to the present disclosure may further include, in one embodiment, a step of receiving a second image corresponding to a product search query from a user terminal (120); a step of generating a search item corresponding to the first image and a search item corresponding to the second image, respectively; and a step of transmitting a search result to the user terminal (120) in response to a determination that there is a search result of a search item corresponding to the first image or a search item corresponding to the second image.

[0096] For example, the search item (640a) may be provided as a search item corresponding to the first image, including a thumbnail image of a representative product image among the search results, search date, time, search progress (%), etc. Such a thumbnail image may serve to visually represent a representative product among the search results, thereby helping the user quickly understand the content or appearance of the product. Alternatively or additionally, the search result may be provided in a form that includes an image (the first image) provided by the user. Additionally, for example, the search item (640b) is a search item corresponding to the second image.

[0097] The product search result user interface (600) may provide a function to add / delete new search items as needed. For example, the product search result user interface (600) may be provided so that the user can customize the search items as needed, such as a 'take a picture' option (610) that receives an image in real time from the user terminal (120), an 'upload image' option (620) that receives an image previously stored in the storage of the user terminal (120), or a 'delete' option (630) that deletes search items that the user no longer needs to search.

[0098] Search items (640a, 640b) may display information on the progress of a search by the electronic device (110) as a percentage (650a). Additionally or alternatively, for example, when an image search is complete, instead of displaying a percentage of 100%, a "View Search Results" function may be provided (650b). Additionally, even when an image search is in progress, image search results may be provided as the search progresses (660). For example, even when "Image Search in Progress (50%)" is displayed, search results (670, 680) obtained up to that point may be provided to the user terminal (120) upon the user's request.

[0099] Those skilled in the art will understand that the above-described configuration enables asynchronous search, i.e., search results are provided after a certain amount of time has elapsed since the user terminal provided the image, rather than simultaneously with the image being provided. Furthermore, the present disclosure allows for sufficient time to perform searches on an expanded data set using various algorithms, freeing users from real-time time constraints.

[0100] According to the present disclosure, more accurate image retrieval can be achieved by using one or more, more preferably two or more, of image retrieval algorithms such as CNN (Convolutional Neural Network), ResNet (Residual Network), and other algorithms that are difficult to use in real-time image retrieval but require a long time, and the like.

[0101] In one embodiment, the step of transmitting the results of a product search to a user terminal (120) may include the step of determining the priority of products in the search results based on a product review rate; and the step of transmitting the search results together with the priority to the user terminal (120).

[0102] Additionally or alternatively, in one embodiment, the step of transmitting the results of a product search to a user terminal (120) may include the step of determining a priority of a product in the search results based on a product review rate; and the step of transmitting the product search results to the user terminal according to the priority.

[0103] FIG. 7, FIG. 8a, FIG. 8b, FIG. 8c and FIG. 8d are flowcharts illustrating in detail the steps of a method according to one embodiment of the present disclosure.

[0104] FIG. 7 is a flowchart generally illustrating detailed steps of a method according to an embodiment of the present disclosure. The detailed steps of the method (S700) according to an embodiment of the present disclosure include a step of receiving a first image from a user terminal (120) (S710), a step of providing an image quality check (S720), a step of performing an image search by an electronic device (110) (S730), and a step of providing the search results to the user terminal (120) (S740). Each step is described in detail below with reference to FIGS. 8a-8d.

[0105] Figure 8a illustrates a step (S710) of receiving a first image from a user terminal (120). Step (S710) may include steps (S811) to (S814) in detail.

[0106] In step (S811), an image search process is initiated. The user terminal (120) may receive a decision from the user regarding whether to use an image previously stored in the storage as input (S812). In response to a decision not to use a previously stored image as input, the electronic device (210) may receive an image acquired in real time through the input unit of the user terminal (120). Additionally or alternatively, in response to receiving a decision to use a previously stored image as input, the electronic device (210) may receive an image previously stored in the storage unit of the user terminal (120).

[0107] In one embodiment, the electronic device (110) may exclude portions of the first image received from the user terminal (120) that may act as noise in image search. For example, if the product corresponding to the query is a hat, and the received image includes the hat and a woman's top, the portion of the woman's top may be determined as noise and excluded from the image. Furthermore, according to an additional embodiment of the present invention, the importance of recognizable objects in the first image may be calculated by considering the position or shape of the corresponding object in the first image, and exclusion processing may be performed based on the importance. For example, if the hat is located in the center of the first image and the woman's top is located on the outer side of the first image, the woman's top may be determined as noise and excluded.

[0108] In one embodiment, the electronic device (110) may receive a first image and, if the first image includes multiple objects, generate search items for each of the objects. For example, if the first image includes a hat and a women's top, product search items may be generated for each of the hat and the women's top.

[0109] In one embodiment, if the first image is an image pre-stored in the user terminal (120) and the electronic device (110) has been granted access to the pre-stored image in advance by the user, the electronic device (110) can search for a product corresponding to the pre-stored image without an explicit search command from the user. For example, if an image of a hat is pre-stored in the user terminal (120) and the electronic device (110) has been granted access to the image in advance by the user, the electronic device (110) can create a product search item for the hat image even without an explicit user input to search for a product corresponding to the image.

[0110] Additionally or alternatively, in one embodiment, the electronic device (110) searching for a product corresponding to the stored image without an explicit search command from the user may include searching for the product based on purchase history information.

[0111] In one embodiment, the electronic device (110) may transmit the results of a product search to the user terminal (120) as a push alarm. For example, if a search result for a product search item for a hat is derived, the electronic device (110) may provide this to the user terminal (120) in the form of a push alarm.

[0112] In one embodiment, the step of receiving the first image may further include, in response to receiving the first image, applying post-processing imaging processing to the first image. In one embodiment, the post-processing imaging processing may include cropping or blurring based on the product included in the first image.

[0113] Figure 8b illustrates a step (S720) for providing an image quality check. Step (S720) may include steps (S821) to (S824) in detail.

[0114] The electronic device (110) may execute an image quality check (S821). In response to a determination that the result of the image quality check is a pass, the electronic device (110) may upload one or more images received from the user terminal (120) to a search item (S822). Additionally or alternatively, in response to a determination that the result of the image quality check is a fail, the electronic device (210) may provide an image quality guide to the user terminal (120) (S823).

[0115] In one embodiment, the step of providing an image quality guide may further include a step of providing at least one guideline for image capturing to the user terminal (120). Specifically, providing at least one guideline for image capturing may mean presenting a capturing range and / or capturing content as a guideline during image capturing. For example, the guideline may be provided by presenting a reference line for optimal image capturing to an input unit (e.g., a camera) of the user terminal (120), similar to capturing necessary documents such as insurance policy terms. Additionally or alternatively, in one embodiment, the step of providing an image quality guide may further include a step of reacquiring an image in real time through the user terminal (120).

[0116] In one embodiment, the at least one guide line may be at least one of a geometric shape representing an imaging range for imaging at least one of a shape of the product, a product barcode, and product information.

[0117] Figure 8c illustrates a step (S730) of searching for a product corresponding to a received image. Step (S730) may include steps (S831), (S834), and (S835) in detail.

[0118] In one embodiment, the electronic device (110) may compare the received image with product description images and / or product review images in a database within the electronic device (110) to determine, as search results, products whose feature information in the images is identical or similar or whose label configurations correspond to each other. There may be at least one such search result.

[0119] The electronic device (110) can start image search based on the received image (S831).

[0120] However, although not illustrated in FIG. 8C, prior to step S831, the electronic device (110) may collect product management information. For example, the electronic device (110) may be equipped with a crawler, a parser, and an indexer to collect web documents on e-commerce and access text information such as product description images, product names, prices, and product review images contained in the web documents. For example, the crawler may collect a list of web addresses on e-commerce and transmit data related to product information to the electronic device (110) by tracking links. At this time, the parser may interpret the web documents collected during the crawling process to extract product management information such as product description images, product prices, product names, product review images, and product review information contained in the pages, and the indexer may index the corresponding locations and meanings.

[0121] In one embodiment, the electronic device (110) may receive a first image corresponding to a product search query and compare the received first image with a product description image database (832) and / or a product review image database (833). In one embodiment, the electronic device (110) may perform the comparison based on deep learning technology in conjunction with an image deep-learning neural network.

[0122] Specifically, after preprocessing (S834) images of the product description image database (832) and / or the product review image database (833) as required for each algorithm, a search may be performed from the product description image (832) and / or the product review image (833) by applying one or more algorithms, more preferably two or more algorithms (S835). The applicable algorithms in step S835 may include various deep learning algorithms of Image Classification on ImageNet. For example, one or more of CNN-based VGGNet, AlexNet, ResNet, GoogLeNet, NFNet, etc. may be included.

[0123] In step S835, the electronic device (110) may perform a search by extracting a label. Extracting a label may be a method of extracting semantic information within an image and performing an image search based on the extracted information. More specifically, the electronic device (110) may use object detection technology among deep learning technologies to recognize various objects within the received image as products and extract features of the products, thereby extracting them as labels. For example, a CNN, more preferably a ResNet, may be additionally applied to perform such object detection and feature extraction within the image. Such label search and CNN or ResNet search may be applied separately or, preferably, together.

[0124] In one embodiment, the electronic device (110) may perform operations for image processing techniques such as object detection, segmentation, inpainting, feature map extraction, feature vector detection, text localization, and / or text segmentation based on the first image.

[0125] In one embodiment, the electronic device (110) can generate search results by calculating similarity between images. Specifically, the electronic device (110) can extract image features using a CNN-based ResNet and calculate similarity between images based on the extracted features to generate search results. For example, the electronic device (110) can embed images into vectors using a pre-trained CNN or ResNet, and then calculate the similarity between a new image and the embedded image to retrieve the most similar image.

[0126] Those skilled in the art will understand that the above-described configuration makes it difficult to apply to real-time search due to the long processing time. Furthermore, those skilled in the art will understand that the above-described configuration allows the image sets within the database (732, 733) to be utilized as large-scale big data. Therefore, the present disclosure enables more accurate image search than conventional real-time image search.

[0127] In one embodiment, the electronic device (110) learns the characteristics of images corresponding to each label to form an initial neural network model, and may further refine the neural network model by applying a large number of product description images and / or product review images. Furthermore, if the product is not included in any group, the electronic device (110) may create a new group that includes the product.

[0128] When using such deep learning technology, the electronic device (110) can collect feedback on search results and update the distance and weight values ​​of layers using the feedback information. That is, when feedback is provided on the classification of product groups and / or the suitability of the product composition included in the product groups, the parameters or weights of the learning model can be updated using this.

[0129] In one embodiment, the method may further include the steps of: determining whether a first image includes images of multiple products; and, in response to determining that the first image includes images of multiple products, generating a search item corresponding to each image of the multiple products. In a further embodiment, the user terminal (120) may receive a decision from the user regarding whether to search for multiple products using a single image. In this case, the electronic device (110) may prioritize products or search for products using one or more algorithms without being subject to the time constraints of real-time search. Specifically, the priority of products may be calculated based on the size or degree of focus of an object within the image.

[0130] In one embodiment, the method may include providing a Quality of Service (QoS) function based on image search performance. This QoS function may provide an electronic device (110) with faster and more accurate search capabilities. Specifically, the QoS function includes functions such as bandwidth enhancement, transmission reliability, and real-time performance to support services related to the electronic device (110) with various requirements.

[0131] FIG. 8d illustrates terminating image search by providing search results to search items (S740).

[0132] The electronic device (110) may provide search results to the search item (S841). Providing the search results to the search item (S841) may mean that the electronic device (110) transmits the product search results to the user terminal (120). According to this provision / transmission, when the user accesses the search item, the product search results may be displayed on the user terminal (120).

[0133] In one embodiment, the step of transmitting the results of a product search to a user terminal (120) may include the steps of: collecting product review rates in association with an image set; determining a priority of the image set based on the collected product review rates; and transmitting the search results together with the priorities to the user terminal (120).

[0134] In the flowcharts according to the present disclosure, each step of the method or algorithm is described in a sequential order. However, the steps may be performed in any order that can be arbitrarily combined, in addition to being performed sequentially. The description of the flowcharts or flowcharts of the present disclosure does not exclude changes or modifications to the method or algorithm, and does not imply that any step is essential or desirable. In one embodiment, at least some of the steps may be performed in parallel, iteratively, or heuristically. In another embodiment, at least some of the steps may be omitted, or other steps may be added.

[0135] Various embodiments according to the present disclosure may be implemented as software on a machine-readable storage medium. The software may be software for implementing various embodiments described in the present disclosure. The software may be inferred from various embodiments described in the present disclosure by programmers skilled in the art to which the present disclosure pertains. For example, the software may be a program including machine-readable instructions (e.g., instructions, codes, or code segments). The device may be a device capable of operating according to instructions called from a storage medium, such as a computer. In one embodiment, the device may be a computing device according to various embodiments described in the present disclosure. In one embodiment, the processor of the device may execute the called instructions, causing components of the device to perform functions corresponding to the instructions. The storage medium may refer to any type of recording medium that stores data and can be read by the device. The storage medium may include, for example, ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, etc. In one embodiment, the storage medium may be implemented in a distributed form, such as in a network-connected computer system. In this case, the software may be distributed, stored, and executed in the computer system. In another embodiment, the storage medium may be a non-transitory storage medium. A non-transitory storage medium refers to a tangible medium that exists regardless of whether data is stored semi-permanently or temporarily, and does not include signals that are transmitted transitively.

[0136] While the technical concepts of the present disclosure have been described through various embodiments, it should be understood that the technical concepts of the present disclosure encompass various substitutions, modifications, and variations that can be made within the scope of those skilled in the art to which the present disclosure pertains. Furthermore, it should be understood that such substitutions, modifications, and variations are encompassed within the scope of the appended claims.

Claims

1. In a product search method performed by an electronic device, A step of receiving a first image corresponding to a product search query from a user terminal; In response to receiving the first image, a step of searching for a product corresponding to the first image based on a set of images, wherein the set of images includes at least one of a product description image and a product review image; and A method comprising a step of transmitting the results of the product search to the user terminal.

2. In paragraph 1, A method wherein the first image includes at least one of an image acquired in real time through an input unit of the user terminal and an image previously stored in a storage unit of the user terminal.

3. In paragraph 1, A method characterized in that the above product review image is updated in real time or periodically by the electronic device.

4. In the first paragraph, the method, A step of receiving a second image corresponding to a product search query from a user terminal; A step of generating a search item corresponding to the first image and a search item corresponding to the second image, respectively; A method further comprising the step of transmitting a search result to the user terminal in response to a determination that there is a search result of a search item corresponding to the first image or a search result of a search item corresponding to the second image.

5. In the first paragraph, the method, A step of executing an image quality check of the received first image; and A method further comprising the step of providing an image quality guide to the user terminal when the result of the image quality check is determined to be failed.

6. In paragraph 5, The steps to provide the above image quality guide are: A method further comprising the step of providing at least one guideline for image capturing to the user terminal, wherein the at least one guideline is at least one of geometric shapes indicating a capturing range for capturing at least one of a shape of a product, a product barcode, and product information.

7. In paragraph 1, The first image is an image previously stored in the user terminal, and the electronic device is granted access to the previously stored image. A method characterized in that, when it is determined that the above access right exists, a product corresponding to the first image is searched for without an explicit search command from the user.

8. In paragraph 1, A method characterized in that the results of the above product search are transmitted as a push alarm.

9. In the first paragraph, the method, A step of determining whether the first image includes images of multiple products; and In response to determining that the plurality of images of the above products are included, a step of generating a search item corresponding to each image of the plurality of products. A method further comprising:

10. In paragraph 1, The step of transmitting the results of the above product search to the user terminal is: A step for determining the priority of products in search results based on product review rates; A step of transmitting the product search results along with the above priorities to the user terminal. A method comprising:

11. In paragraph 1, The step of receiving the first image comprises: In response to receiving the first image, a step of applying post-processing imaging processing to the first image. How to include more.

12. In paragraph 11, The above post-processing imaging processing is, A method comprising cropping or blurring based on a product included in the first image.

13. In a product search method performed by a user terminal, A step of acquiring a first image corresponding to a product search query and transmitting the image to an electronic device; In response to receiving the first image from the electronic device, a step of receiving a search result of a product corresponding to the first image based on a set of images, wherein the set of images includes at least one image from among a product description image and a product review image. A method comprising:

14. In paragraph 13, A method wherein the first image includes at least one of an image acquired in real time through an input unit of the user terminal and an image previously stored in a storage unit of the user terminal.

15. In electronic devices, One or more processors, comprising one or more memories storing instructions executed by the one or more processors; An electronic device, wherein when the instructions are executed by the one or more processors, the one or more processors are configured to execute a method according to any one of claims 1 to 14.

16. In a non-transitory computer-readable recording medium recording instructions that, when executed by one or more processors, cause the one or more processors to perform an operation, A non-transitory computer-readable recording medium, wherein the instructions are configured to cause the one or more processors to execute a method according to any one of claims 1 to 14.

Images