IMAGE-BASED PROPOSAL GENERATION SYSTEM

Abstract

This invention relates to a system (1) that analyzes the product, object and usage environment in a photograph selected by the user, determines the user's purchase needs, and offers product, payment, financing and insurance options suitable to those needs.

Description

1 TARIFF IMAGE-BASED PROPOSAL GENERATION SYSTEM Technical Area This invention features products, objects, and uses included in a photograph selected by the user. By analyzing the environment, it identifies the user's purchasing needs and responds to those needs. a system that enables the provision of suitable product, payment, financing and insurance options It is related to. Previous Technique Today, visual search and image recognition technologies are used. The products or objects in the images are identified, similar products are found, and Search or purchase processes can be initiated based on visual intent. Currently, there are 15 options. When evaluating product stock, price, and delivery information in systems, financing is considered. Options depend on credit and installment suitability, while insurance options depend on coverage and They are determined separately according to the risk class. However, intent and The level of trust is assessed together with the suitability requirements of these different services. It cannot be combined into a single offer package. Therefore, unnecessary service 20 conducting inquiries, generating inappropriate offers, and user selection. Subsequently, deficiencies emerged such as the need to repackage the product. It is emerging. Therefore, considering the studies and shortcomings in the current technology, 25 When available, the product is obtained from the image selected by the user. Identifying user needs through usage information and providing products suitable for those needs. Checking the compatibility of financing and insurance options and by eliminating unsuitable or insufficiently reliable options, the valid ones are determined. 30 It is understood. 2 Chinese patent CN120235675A, which is included in the known state of the art. The document allows users to search for products using text or image input. its intention is to showcase large, multi-modal models, along with appropriate product images and specifications. a platform that creates and presents matching products to the user on an e-commerce platform The system is being discussed. The invention in question is located on the server of the e-commerce platform. or a plugin running on the user's device or directly integrated It can be implemented as a search module. When the user enters the search interface... Initial product attributes to be searched from text or uploaded image. These initial features are extracted by feeding them into the query generation model for semantic analysis. New product features and candidate search areas are being created that are related to this aspect. 10 As a result of expanding product features, the user can access not only what they say, but also... Product search intention: not based on the features it displays, but on the actual product the user wants to find. This is determined. The determined product search intent is transferred to the producer model, and the candidate is identified. Images representing the products, feature labels, and structured product information. Parameters are being created. The user selects the generated candidate image and features. 15 The user can select the option closest to the product they are looking for from among the pairs via the interface. They can choose. The selected product description or image will be displayed on the e-commerce platform. Product titles, visual features, page parameters, and user interaction. It is compared with the information. The match value with the search intent is a certain threshold. Products exceeding the value are identified and ranked according to their matching degree, and 20 is shown to the user. Brief Description of the Invention The purpose of this invention is to identify the product, object, and 25 items in the image selected by the user. By analyzing the usage environment, we can identify the product the user might want to buy and Identifying the payment, financing, and insurance needs for the product, the selected product stock, price, campaign, financing availability and insurance coverage information. By evaluating the suitable options, we create and display them as a single offer package. If the analysis is insufficient, the offer will be adjusted according to additional information received from the user. The goal is to implement a system that reorganizes the package. 3 Detailed Description of the Invention The "Image-Based Proposal" implemented to achieve the purpose of this invention. The "Generation System" is shown in the attached figure; Figure 1. Schematic view of the system described in the invention. The parts shown in the figure are individually numbered, and the corresponding numbers correspond to these numbers. given below: 1. System 2. Application 3. Database 4. Server The product, object, and usage environment featured in a photo selected by the user. By analyzing the user's needs, it identifies their purchasing needs and provides solutions tailored to those needs. to provide product, payment, financing and insurance options The developed invention subject system (1); - the user's smartphone, tablet computer, desktop computer or portable 20 an electronic device in the form of a computer that automatically processes data. running applications and / or software that function and produce meaningful results, the user selects at least one photo from their gallery, and depending on the selected photo The system displays and verifies the created product, financing, and insurance options. They should answer the questions and make a choice, accept or reject the proposed package of proposals. at least one configured to enable it to perform rejection operations application (2), - product catalog records, stock status, price range, campaign rules, delivery terms and conditions and product compatibility information, financing and insurance policy versions, model versions, risk thresholds, audit trail logs, and user interaction with 30 4 to store feedback data obtained from the processing results at least one database structured (3) and - Image embedding, object, scene, image quality from the photo selected by the user, Identifying information on trust and uncertainty, and contextualizing this information with contextual data. by evaluating product class, use case, price range, protection needs and 5 To create an intent vector that includes funding sensitivity, based on the intent vector. Identifying product, financing and insurance candidates, evaluating candidates based on prerequisites, risk contribution and converting proposal atoms carrying suitability information, constraint graph of proposal atoms and to create a feasible proposal package by evaluating it within the scope of the risk budget. and if the risk budget is exceeded, 10 according to the verification response received from the user. At least one server configured to regenerate the offer package (4) It includes. The application (2) in the system (1) that is the subject of the invention, the user's electronic device 15 by the server (4) to choose at least one photo from the gallery. The created product, financing, and insurance options are all integrated into a single user flow. within it, answer verification questions, and add to the offer package. at least the amount that enables the user to make selections, acceptances, or rejections. It is structured to provide an interface. The database (3) in the system (1) which is the subject of the invention contains the catalog records of the products, Stock availability, price range, campaign rules, delivery conditions, and product compatibility. information; eligibility class used in financing transactions, installment options Information regarding the status, limit range, offer display permission, and financing policy version; Product class, coverage template, exclusion rules used in insurance transactions, 25 record risk level, policy display restrictions, and insurance policy release information. It is structured to keep it under control. The database (3) is obtained from the image. with the model version used in generating the intent vector and proposal package risk thresholds, prerequisites for the generated proposal atoms, risk contribution, suitability mask, validity period, service call cost, description code and return 30 cost information; decision time, intent dimensions used, rejected bid component. and audit trail records in the form of the technical code of the elimination justification; user interaction, incorrect match notification, package cancellation, re-selection after product selection Compilation results, financing application results, and insurance purchase results. This feedback data is combined with the relevant model and policy versions. storing them by associating them and including those records in the offer packages 5 evaluation, retrospective review of decisions, model calibration, and to ensure its use in policy improvement processes It is being structured. The server (4) in the system (1) which is the subject of the invention, any remote communication 10 to communicate with the application (2) and the database (3) using the protocol and In order to exchange data with the application (3) through this established communication is configured. The server (4) opens the user's account via the application (2). the gallery photo and time range selected for initiating the process, application session, Permitted context data in the form of approximate usage context and seasonal information 15 taking, or pre-processing the image on an electronic device, is called a raw photograph. instead, anonymized image embedding, category possibilities, uncertainty range, and By obtaining the minimum contextual attributes necessary for the transaction, the user's raw personal or It is configured to prevent the transmission of financial data. Server (4), If the raw photograph is taken, the image quality, scale, orientation and content will be adjusted accordingly. normalizing in terms of density, the objects within the image and their characteristics Identifying the product subcategories to which it may belong, and the image's context of use. extracting the scene classes that represent the digital characteristics of the image to create the visual evidence and demonstrate the reliability of these inferences 25 to produce image quality indicator, detection confidence and uncertainty values. Convolutional Neural Network (CNN), EfficientNet Network), YOLO (You Only Look Once), DETR (Detection Network), Transformer – Detection Transformer), Vision Transformer or CLIP (Contrastive Language-Image Pre-training) At least one of the visual analysis methods (Image Pre-training) is required by 30. It is configured to use. The server (4) obtains the visual from the image. 6 Burial, object and scene possibilities, as well as image quality and reliability values, are permitted. Multimodal visual with context data and catalog information registered in the database (3) To evaluate together within the framework of the intention model, this evaluation involves multiple stakeholders. neural networks, gradient boosting models, factorization machines, transformer-based multi-mode models or caliber 5 to utilize established logistics models and understand the user's purchasing intent; The possibilities of the product identified in the photograph belonging to different product classes, the product the possible use scenarios for which it will be used, the estimated price for the product. the time interval, the period for the short-term or long-term use of the product proximity, the level of need for protection of the product, payment or financing 10 the level of need for the option, the result obtained from image analysis the level of confidence, the range of uncertainty associated with this result, and the product, financing or insurance transactions are available under current conditions, preliminary information or a multi-dimensional document containing transaction permissions indicating that it is not visible It is structured to be generated in the form of an intent vector. Server (4), 15 The generated intent vector and visual embedding are registered in the database (3) product catalog data, stock availability, price range, campaign rules, delivery conditions and product Comparing with compatibility information to visual similarity, category fit, price range, Identifying product candidates based on stock and campaign conditions; intent vector and The product price range is determined by the financing service provider using a raw 20. Instead of financial data, the returned eligibility class, installment status, and limit are displayed. range, financing policy version and offer display permission information by evaluating and generating installment, credit or payment plan candidates; product class, Use case, protection need score, risk class, coverage template, exclusion By evaluating the rules and policy display restrictions, we identify insurance candidates and 25 The product, financing, and insurance options offer the same results instead of independent outcomes. It is structured to be generated based on intent data. Server (4) presents product, financing and insurance candidates with different data structures. To enable comparison, each of the candidates shared a common technical data structure. transforming the offer into an atom representing the offer, with each offer atom containing 30 a prerequisite that must be met for it to be used is the uncertainty in image analysis. 7 and the added risk arising from other eligibility conditions, to the use of the offer or the compliance mask, which determines whether or not it is permitted to be displayed, validity period, cost of the relevant service call, selection or rejection of the offer the explanation code that allows the reason to be traced and the subsequent withdrawal of the offer In this case, assign the resulting recovery cost and thus from 5 different services. to make the obtained candidates usable in a common package creation process It is structured as follows: The server (4) distributes the generated offer atoms to the node, product, compatibility and dependency relationships between financing and insurance offer atoms Creating a constraint graph that defines the relationship, displaying stock, price, etc. on the graph. campaign, financing eligibility, insurance coverage and exclusion conditions, 10 regulatory constraints, model confidence threshold, user validation status, and suitability. evaluating their masks, the total consisting of the risk contribution of each atom of proposal decision risk is defined as an acceptable upper limit predetermined for the system. comparing with the risk budget and constraint programming, graphical-based optimization, score-based rule engine, mixed integer optimization, integer linear 15 mandatory by using programming or risk-weighted scoring methods meeting the eligibility criteria and having a total risk below the risk budget Among the combinations, the one that offers the highest expected package benefit is the feasible one. The product is configured to select the financing and insurance package. Server (4), The confidence level of the inference drawn from the image is insufficient, the total risk is risk 20. a particular product, financing or insurance may be overlooked or oversold due to uncertainty. If the proposed atoms cannot be enclosed in a binding package, the process can be carried out directly. Instead of terminating the process, what is the current uncertainty regarding additional information that can be obtained from the user? Information gain, which is said to reduce packet risk to a significant extent, is expected to create risk. expected risk reduction, indicating a decrease, and the user's additional question due to 25 by evaluating the interaction cost, which represents the processing load the package will encounter. Identifying the verification question that will minimize the risk the most, and then asking the selected question. transmitting to the user via application (2), received via application (2) verification response or intent vector after user product selection update, recalculate the risk budget, bid atoms and constraint graph 30 reassessment and only products that pass the compliance mask, funding and 8 The current offer package created from insurance offers is available on the application (2) Two-stage package compilation for display in user feed. It is configured to perform the operation. Server (4), application (2) Product selection obtained through this process, incorrect match notification, package rejection or cancellation, Following the product selection process and subsequent reassessment, financing was determined as follows: 5 feedback data in the form of application results and insurance purchase results data by associating it with the relevant intent vector, model version, and policy version. to record in the base (3); the image features are located in the intent vector product class, use case, price range, protection needs, and financing. the effects on sensitivity, the stock, price, financing of the proposed atoms 10 suitability, insurance coverage, campaign, regulatory constraints, model confidence, and Depending on user verification conditions, the reasons for elimination and the decision will be explained. The model used in its creation and policy versions are presented as audit trails. to store; new model or policy releases to a limited group of users By running it, the invalid package rate, verification requirement, and offer rejection rate are 15. monitoring results such as model confidence distribution, mismatch rate, invalid packet rate and changes occurring over time in feedback records data / model distribution bias, population stability index and specified Identifying performance indicators and, if necessary, implementing an intent model. calibration, risk budget thresholds, proposed atomic weights, or model 20 to ensure that the processes for retraining are updated It is being structured. Industrial Application of the Invention Thanks to the system (1) that is the subject of the invention, e-commerce platforms can make online shopping possible. digital applications where digital finance and insurance services are offered together on platforms, the product, environment, or use case selected by the user Analyzing an image to determine the user's potential needs. and products, payment or financing options, and related 30 that meet this need. 9 Insurance offers are evaluated together and a single offer package is presented to the user. It is ensured that it is presented in this way. Based on these fundamental concepts, the invention is a "Image-Based Proposal Generation System". It is possible to develop a wide variety of applications related to (1)”, and the invention here is 5 It cannot be limited to the examples given; it is essentially as stated in the claims.

Claims

REQUESTS 1. Product, object, and use featured in a photo selected by the user. by analyzing the environment, it determines the user's purchasing needs and 5. To offer products, payment, financing, and insurance options tailored to needs. providing; - the user's smartphone, tablet computer, desktop computer or run on an electronic device in the form of a portable computer, an application that automatically processes data and produces meaningful results and / or execute software that takes at least one photo from the user's gallery. the selection, product, financing and insurance created according to the selected photo display the options, answer the verification questions, and selection, acceptance or rejection processes for the proposed offer package at least one application (2) designed to enable it to perform - product catalog records, stock status, price range, campaign rules, 15 delivery terms and product compatibility information, financing and insurance. policy versions, model versions, risk thresholds, audit trail records and feedback obtained from user interaction and transaction results at least one database configured to store feed data (3) containing and 20 - visual embedding, object, scene, image from a photo selected by the user Identifying information on quality, reliability, and uncertainty, and contextualizing this information. By evaluating the data, product class, use case, price range, intent vector including protection needs and funding sensitivity to create, product, financing and insurance candidates according to the intent vector 25 to identify candidates, proposals containing prerequisite, risk contribution and eligibility information converting into atoms, proposal atoms constraint graph and risk budget by evaluating the situation and creating a feasible proposal package and taking risks If the budget is exceeded, according to the verification response received from the user. At least one server configured to regenerate the offer package (4) 30 a payment system characterized by (1). 11 2. At least one item from the user's gallery on their electronic device the selection of the photo, product, financing and created by the server (4) to display insurance options within a single user flow, answering verification questions and making a choice or acceptance regarding the offer package or at least one interface that enables it to perform rejection operations 5 The application (2) is characterized by the Request structured to deliver A system like the one in 1 (1).

3. Product catalog records, stock status, price range, and campaigns. rules, delivery conditions and product compliance information; financing 10 eligibility class used in transactions, installment status, limit range, offer display permission and financing policy release information; insurance Product class used in transactions, guarantee template, exclusion rules, risk level, policy display restrictions, and insurance policy release information. 15 characterized by the database structured to keep records (3) a system like any of the above-mentioned requests (1).

4. The intent vector and proposal package obtained from the image. the model version used in its creation and the risk thresholds, created Proposal requirements, risk contribution, conformity mask, validity 20 Information including duration, service call cost, description code, and recovery cost; decision time, dimensions of intent used, eliminated bid component, and elimination. the audit trail records in the form of a technical code for the justification; user interaction, incorrect match notification, package cancellation, post-product selection Result of reassessment, result of financing application and insurance purchase 25 The feedback data in the form of results can be used in the relevant model and policy. to store and offer those records by associating them with the versions. evaluation of packages, retrospective review of decisions, to be used in model calibration and policy improvement processes 30 characterized by the database structured to provide (3) a system like any of the above requests (1). 12 5. Using any remote communication protocol, the application (2) and data to communicate with the base (3) and through this communication to the application (3) server (4) configured to exchange data with a system like any of the above characterized claims (1). 5 6. The application (2) selects the user to initiate an open process. gallery photo and time range, application session, approximate usage To retrieve permitted context data in the form of context and season information, If the image is pre-processed on an electronic device, the raw photograph becomes 10. instead of anonymized visual embedding, category possibilities, ambiguity By retrieving the range and the minimum context attributes required for the operation, the user to prevent the transfer of raw personal or financial data from the above requests characterized by the configured server (4) a system like any other (1). 15 7. If a raw photograph is obtained, the image's quality, scale, orientation, and content can be adjusted. normalizing in terms of density, the objects within the image and Identifying the product subcategories to which these may belong, the image extracting scene classes that represent the usage environment, image 20 to create a visual embed representing its numerical properties and this Image quality indicator to show the reliability of inferences, Convolutional neural network to generate perception confidence and uncertainty values, EfficientNet, DETR, visual converter, or CLIP form of visual analysis Server (4) configured to use at least one of the methods 25 a system like any of the above characterized claims (1).

8. Image embedding, object and scene possibilities derived from the image. quality and trust values, permission context data and database (3) 30 Within the framework of a multimodal visual-intent model with registered catalog information. 13 to evaluate together, multitasking neural networks in this evaluation, gradient boosting models, factorization machines, converter-based utilizing multimodal models or calibrated logistics models and the user's purchase intention; the product identified in the photograph is different. the likelihood of belonging to product classes, the use case in which the product will be used 5 the probabilities of the scenarios, the estimated price range for the product, the product proximity in time for use in the short or longer term, the level of need for protection of the product, payment or financing the level of need for the option, obtained from image analysis the reliability level of the result, the uncertainty range for this result, and the product, 10 financing or insurance transactions are available under current conditions, prior transaction permissions indicating that it is not informative or cannot be displayed to create it in the form of a multidimensional intention vector containing from the above requests characterized by the configured server (4) a system like any other (1). 15 9. The generated intent vector and visual embedding are registered in the database (3) as products. Catalog data, stock availability, price range, campaign rules, delivery. visual similarity by comparing conditions and product compatibility information, Depending on category suitability, price range, stock and campaign conditions, product 20 Identifying candidates; financing the intent vector and product price range. instead of raw financial data by the service provider that provides the service returned eligibility class, installment status, limit range, along with financing policy version and offer display permission information. by evaluating and creating installment, credit or payment plan candidates; product 25 class, use case, protection need score, risk class, collateral by evaluating the template, exclusion rules and policy display restrictions Identifying insurance candidates and determining product, financing and insurance options instead of independent results, based on the same intention data. 30 characterized by the server (4) configured to enable its creation a system like any of the above-mentioned requests (1). 14 10. Product, financing and insurance candidates with different data structures To enable comparison, each of the candidates was assigned common technical data. to transform the structure into a proposal atom, each proposal atom The prerequisite that must be met in order for the offer to be used is the image. Risk arising from uncertainty in the analysis and other eligibility conditions 5 his contribution, whether the offer is allowed to be used or displayed the suitability mask indicating that it was not given, its validity period, relevant the cost of the service call, the reason for the selection or rejection of the offer the disclosure code that enables tracking and subsequent withdrawal of the offer assigning the resulting buyback cost and thus different 10 in the process of creating a common package of candidates obtained from the services characterized by the server (4) configured to make it available a system like any of the above-mentioned requests (1).

11. The generated offer atoms are node, product, financing and insurance offer 15 the compatibility and dependency relationships between atoms as links Creating a constraint graph that defines stock, price, campaigns, etc., on the graph financing eligibility, insurance coverage and exclusion conditions, regulatory constraints, model confidence threshold, user validation status, and suitability. evaluating their masks, 20 consisting of the risk contribution of each atom of proposal total decision risk is limited to a predetermined acceptable upper limit for the system. comparing it with the risk budget and constraint programming, graphic Rule-based optimization, score-based rule engine, mixed integer optimization, integer linear programming or risk-weighted scoring using these methods, meeting the mandatory compliance requirements and 25 from combinations where the total risk is below the risk budget The feasible product, financing, and package that offers the highest expected benefit. Characterized by the server (4) configured to select the insurance package a system like any of the above-mentioned requests (1). 15 12. Insufficient confidence in the inference drawn from the image, overall risk exceeding the risk budget or certain product due to uncertainty The inability to include financing or insurance proposals in the binding package. Instead of directly terminating the process in this case, additional funds can be obtained from the user. Information gain, which expresses the extent to which information reduces existing uncertainty, 5 expected risk refers to the anticipated reduction in package risk. reduction and the processing load the user will encounter due to additional questions by evaluating the cost of the interaction, whichever minimizes packet risk. Determine the verification question, determine the question through the application (2) to transmit to the user, the verification response received via application (2) or 10 Updating the intent vector after the user selects a product, risk recalculating the budget, redesigning the proposal atoms and the constraint graph. to evaluate and only select products that pass the suitability assessment, funding and Applying the current offer package created from insurance offers (2) transmitted to be displayed as a single-user stream and in two stages 15 with the server (4) configured to perform the package compilation process a system like any of the above characterized claims (1).

13. Product selection obtained via application (2), mismatch notification, 20 Package rejection or cancellation is a re-initiated process following product selection. Compilation results, financing application results, and insurance purchase results. Feedback data in this form is used to generate the relevant intent vector, model version, and to record in the database (3) by associating it with the policy version; The product class included in the intent vector of the image properties, usage 25 scenario, price range, need for protection and financing sensitivity the effects of the offer on stock, price, financing availability, insurance scope, campaign, regulatory constraints, model confidence, and user The reasons for elimination and the decision depend on the verification conditions. The audit trail of policy versions with the model used in its creation is 30. to store; new model or policy versions to a limited number of users 16 By running it in the group, the invalid packet rate, verification requirement, and offer return Monitoring the results in the form of the acquisition rate and the model confidence distribution, incorrect matching rate, invalid packet rate, and time in feedback records. the data / model distribution deviation that occurs within the population Determination based on the stability index and specified performance indicators. 5 by calibrating the intent model and risk budgeting where necessary. thresholds, proposed atomic weights, or model retraining server configured to ensure that processes related to this are updated (4) like any of the above-mentioned claims characterized by system (1). 10 20 30

Images