A Multimodal Unified Material Retrieval Method and System Based on Dual Vector Space
By parallel searching in material vector and tag vector spaces and combining multi-path retrieval results, the problems of missed detection and cold start in existing technologies are solved, achieving efficient and accurate multimodal material retrieval and supporting semantic retrieval for non-text input.
Patent Information
- Application Number
- CN202611353262.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-02
- Publication Date
- 2026-11-17
AI Technical Summary
Existing information retrieval solutions suffer from issues of missed detections and cold start in digital asset management. In particular, high-efficiency retrieval solutions struggle to handle user-maintained tag systems and semantic relationships, resulting in low retrieval efficiency.
A multimodal unified material retrieval method based on dual vector space is adopted. By receiving multimodal retrieval requests, identifying and selecting corresponding search strategies, executing single-path or multi-path retrieval tasks, and performing cosine similarity search of material vectors and tag vectors in parallel threads, the results are merged to improve retrieval efficiency and accuracy.
It effectively solves the problems of missed detection and cold start, improves the universality and efficiency of retrieval, and ensures the reliability and accuracy of retrieval results, especially for semantic retrieval capabilities for non-text input.
Smart Images

Figure CN123115442A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of information retrieval technology. More specifically, this invention relates to a multimodal unified material retrieval method and system based on a dual-vector space. Background Technology
[0002] In digital asset management and material management systems, users often need to quickly retrieve materials that match their needs from a massive material library. Mainstream information retrieval solutions are primarily divided into three types: keyword text retrieval, single-vector semantic retrieval, and tag-based filtering retrieval. Keyword text retrieval involves the user entering keywords in the search box, and the system then performs a full-text search on text fields such as material titles and descriptions, returning a list of matching materials. This is the most basic retrieval method, requiring a full-text, item-by-item traversal and matching operation for each retrieval task. If the material library is too large or the returned content is too extensive, the retrieval timeout becomes extremely long, sometimes even triggering a timeout error, making it unsuitable for applications with high real-time requirements.
[0003] Unlike the keyword text retrieval schemes mentioned above, single-vector semantic retrieval involves converting the text description of the material into semantic vectors using a pre-trained language model and storing them in a vector database. When a user queries, the query text is also vectorized, and then the most similar material is retrieved in the vector space using cosine similarity. It relies solely on the material description vectors for similarity matching, making it difficult to utilize the carefully maintained tagging system / favorite annotation information of users. For example, when a user searches for "red dress," a material description might be "an elegant sleeveless knee-length dress in a vibrant Chinese red color." Because the word "dress" doesn't directly appear in the description, the similarity between the material's vector and the query vector might be low, causing the material to be missed. In reality, the material has already been tagged with "red" and "dress" by the user. Tag-based retrieval prioritizes searches based on users' historical tags / favorite tags, significantly improving retrieval efficiency. However, it completely fails to retrieve new materials that haven't been tagged, exhibiting a cold start problem. Furthermore, Boolean logic-based tag filtering cannot understand semantic relationships that are "similar but not identical". For example, when a user selects the "skirt" tag, the system will not automatically associate it with "dress" or "skirt".
[0004] In summary, existing high-efficiency retrieval schemes mainly suffer from missed detections and cold start problems. Summary of the Invention
[0005] To address the technical problems of missed detections and cold start in the aforementioned high-efficiency retrieval schemes, this invention discloses a multimodal unified material retrieval method and system based on dual vector space.
[0006] In a first aspect, the present invention discloses a multimodal unified material retrieval method based on a dual-vector space, comprising:
[0007] Receive multimodal retrieval requests; The multimodal retrieval request is identified, and the identification result is obtained; Based on the recognition results, select the corresponding search strategy; the search strategy shall include at least single vector retrieval and text tag joint vector retrieval. Based on the search strategy, execute a single-path search task or a multi-path search task and return the search results; if a multi-path search task is executed, the search results returned by the multi-path search task will be merged and output.
[0008] Beneficial Effects: After receiving a multimodal retrieval request, the method of this invention first identifies the multimodal retrieval request to achieve retrieval classification; then, based on the identification results, it selects the corresponding search strategy, thereby improving the universality of the method. The search strategy encompasses both single-vector retrieval and text tag joint vector retrieval, thus absorbing the advantages of both strategies while avoiding their inherent disadvantages. During the retrieval stage, single-path or multi-path retrieval is selected based on the search strategy to save computer resources and improve overall efficiency. For multi-path retrieval tasks, the retrieval results returned from multi-path retrieval tasks are merged to prevent missed detections. Compared to existing technologies, the method of this invention can overcome the problems of missed detections and cold start while ensuring efficient retrieval is not compromised.
[0009] Preferably, if the search strategy is text tag joint vector retrieval, a multi-path retrieval task is performed according to the search strategy, and the retrieval results are returned, including: A feature transformation model is used to convert multimodal retrieval requests into query vectors, enabling parallel threads for dual-path retrieval: In the first thread, the query vector is used to perform the first path retrieval task in the preset material vector set, and the first retrieval result is returned; In the second thread, the query vector is used to perform a second path retrieval task in a preset set of tag vectors, and the second retrieval result is returned; The first and second search results are combined and then output.
[0010] Preferably, the first path retrieval task is configured as follows: The query vector is used to perform a cosine similarity search in the material vector set, and the information of the top-ranked materials with the highest similarity and their similarity scores are returned as the first search results.
[0011] Preferably, the second path retrieval task is configured as follows: The query vector is used to perform a cosine similarity search in the tag vector set, and the top similar tags are returned. Retrieve the preset material label association table to find the material ID associated with similar labels; If the same material is matched by multiple similar tags, the highest tag similarity score is taken as the tag path score of the material. The material ID and tag path score are used as the second search result.
[0012] Preferably, the first search result and the second search result are merged and output, including: Record the number of tags that match the material information with similar tags; Calculate the tag hit ratio weight based on the number of tags; Based on the weight of the tag hit ratio, the material similarity score and the tag similarity score are weighted and summed to obtain the comprehensive score; The material information with the highest overall score will be returned as the first search result.
[0013] Preferably, multimodal retrieval requests include, but are not limited to, any one or a logical combination of text keywords, natural query statements, tag ID lists, material address identifiers, images, or videos.
[0014] Preferably, the search strategy also includes vector retrieval based on visual understanding and parallel hybrid retrieval based on logical combination input.
[0015] Preferably, if the recognition result is a material address identifier, the vector retrieval generated based on visual understanding is configured as follows: Determine whether the source URL matches the meta description tag in the database; If so, directly retrieve the meta-description tag and skip the visual understanding and keyframe analysis steps, and obtain the cache file from the cache; If not, perform visual understanding and keyframe analysis tasks to obtain the parsing file online; Generate temporary OSS storage URLs for cached files and / or parsed files; Perform a multi-path retrieval task based on the temporary OSS storage URL, and return retrieval results other than the material address identifier.
[0016] Preferably, if the recognition result is a logical combination, the parallel hybrid retrieval based on the logical combination input is configured as follows: The logic combination is broken down into multiple subtasks that are executed in parallel, and the corresponding sub-search strategies are invoked. Based on the sub-search strategy, single-path retrieval tasks and / or multi-path retrieval tasks are executed in parallel, and the retrieval results are returned.
[0017] Secondly, the present invention discloses a multimodal unified material retrieval system based on a dual vector space, including a processor and a memory. The memory stores computer program instructions, and when the computer program instructions are executed by the processor, the multimodal unified material retrieval method based on a dual vector space described in the first aspect is implemented.
[0018] The advantages of this invention, which differ from existing technologies, are as follows: (1) The method of the present invention supports multimodal input retrieval and schedules the appropriate search strategy to perform single-path retrieval tasks or multi-path retrieval tasks, which has higher universality and is less prone to missed detection or cold start problems.
[0019] (2) For search results based on multiple tags hitting simultaneously, the method of the present invention introduces a comprehensive scoring mechanism, which makes the similarity ranking of the search results more reliable.
[0020] (3) For non-text input retrieval such as images, video URLs or file uploads, the method of the present invention introduces artificial intelligence visual understanding to generate natural language descriptions to support semantic retrieval capabilities for non-text inputs. At the same time, the above process is also combined with a database caching layer for auxiliary retrieval to avoid redundant analysis. Attached Figure Description
[0021] The above and other objects, features, and advantages of exemplary embodiments of the present invention will become readily apparent upon reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of the invention are illustrated by way of example and not limitation, and like or corresponding reference numerals denote like or corresponding parts, wherein: Figure 1 This is a flowchart of the multimodal unified material retrieval method based on dual vector space in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the structure of the multimodal unified material retrieval system based on dual vector space in Embodiment 2 of the present invention. Detailed Implementation
[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0023] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.
[0024] Example 1 like Figure 1As shown, this embodiment discloses a multimodal unified material retrieval method based on a dual-vector space, including: S10: Receive multimodal retrieval requests.
[0025] In this embodiment, multimodal search requests submitted by users are received through a unified search API endpoint. These multimodal search requests include, but are not limited to, any one or a logical combination of text keywords, natural language queries, tag ID lists, material address identifiers, images, or videos. Text keywords are natural language phrases entered by the user in the search box. Natural language queries are complete natural language statements composed of multiple natural language phrases. Tag ID lists can be lists of order IDs, product IDs, or shelf IDs. Material address identifiers generally refer to the network addresses of materials used as the basis for retrieval; similar material resources need to be found during the retrieval process. Image formats are not limited and can be GIF, PNG, or other image formats. Videos are video files uploaded locally or to cloud storage; they also serve as the basis for retrieval, and similar material resources need to be found during the retrieval process. These multimodal search requests can be single-dimensional search requests or logical combinations based on Boolean logic or other combinations.
[0026] S20: Identify multimodal retrieval requests and obtain identification results.
[0027] In this embodiment, it can be determined whether a multimodal search request is a logical combination by detecting specific combination identifiers or descriptions. Specific combination identifiers or descriptions can be "AND", "OR", "and", or "last time", etc. The identifier driving the corresponding search strategy can be defined by the input format or file format of the multimodal search request. For example, if the multimodal search request identifies text keywords or natural query statements, the identifier driving the corresponding search strategy is "TEXT_VECTOR".
[0028] S30: Select the corresponding search strategy based on the recognition results.
[0029] The search strategies include single-vector retrieval, text tag joint vector retrieval, vector retrieval based on visual understanding, and parallel hybrid retrieval based on logical combination input.
[0030] More specifically, the configuration list for the above search strategy can be as follows:
[0031] In a preferred embodiment, the functionalities of the search strategy can be encapsulated in advance, and then the corresponding encapsulated execution function can be called according to the recognition results, thereby achieving adaptive routing and efficient retrieval.
[0032] S40: Based on the search strategy, execute a single-path retrieval task or a multi-path retrieval task and return the retrieval results.
[0033] Specifically, if a multi-path retrieval task is executed, the retrieval results returned by the multi-path retrieval task will be merged and output. Further, for the implementation of the multi-path retrieval task, step S40 above includes: S41: Use a feature transformation model to convert multimodal retrieval requests into query vectors and enable parallel threads for dual-path retrieval.
[0034] In one embodiment, the user-input multimodal retrieval request is converted into a 1024-dimensional query vector using the text-embedding-v4 model.
[0035] S401: In the first thread, the first path retrieval task is performed in the preset material vector set using the query vector, and the first retrieval result is returned.
[0036] The first path retrieval task in step S401 above is configured as follows: The query vector is used to perform a cosine similarity search in the material vector set, and the information of the top-ranked materials with the highest similarity and their similarity scores are returned as the first search results.
[0037] More specifically, the query vector is used to perform a cosine similarity search on the material vector set in Milvus (an open-source cloud-native vector database), returning the top_k most similar material information and material similarity scores.
[0038] The above vector similarity retrieval method effectively improves retrieval return efficiency.
[0039] S402: In the second thread, the second path retrieval task is performed in the preset tag vector set using the query vector, and the second retrieval result is returned.
[0040] The second path retrieval task in step S402 above is configured as follows: First, a cosine similarity search is performed on the tag vector set using the query vector to return several similar tags with the highest similarity. Then, the preset material tag association table is retrieved to query the material ID associated with the similar tags. If the same material is matched by multiple similar tags, the highest tag similarity score is taken as the tag path score of the material. Finally, the material ID and tag path score are used as the second search result.
[0041] The above-mentioned tag retrieval method effectively utilizes the tag vector set to assist in retrieval, thereby avoiding the missed detection problem of pure vector similarity retrieval.
[0042] S42: The first and second search results are merged and output.
[0043] Furthermore, step S42 above includes: S421: Record the number of tags that match the material information with similar tags.
[0044] It's important to explain that a historical ID tag list records the major categories, subgroups, and specific numbers of material information. For example, taking the material information as "red dress," it might be matched with similar tags such as "red," "dress," "clothing," "outfit," and "number." "Clothing" and "outfit" belong to the major categories, while "red" and "dress" belong to the subgroups. All of these tags can match the aforementioned material information.
[0045] S422: Calculate the tag hit ratio weight based on the number of tags.
[0046] The calculation formula for step S422 above is as follows:
[0047] In the formula, This indicates the label hit ratio weight; min represents the minimum value function. This indicates the number of tags that match the material information with similar tags; The total number of tags in the historical ID tag list. In the above formula, the higher the hit rate, the higher the corresponding weight. It should be noted that the maximum weight of the tag hit rate is 1.
[0048] S423: Based on the tag hit ratio weight, the material similarity score and the tag similarity score are weighted and summed to obtain a comprehensive score.
[0049] Specifically, the calculation formula for step S423 above is as follows:
[0050] In the formula, This indicates the overall score; Indicates the material similarity score; This indicates the score for the label path.
[0051] In this way, the weight of dual-path retrieval can be dynamically adjusted. When the tag hit rate is higher, the returned results based on the tag path are more reliable and have a higher corresponding weight. Conversely, when the tag hit rate is lower, the returned results based on the vector retrieval path are higher and have a higher corresponding weight.
[0052] S424: Return the material information with the highest comprehensive score as the first search result.
[0053] Through steps S421-S424 above, when performing a multimodal input retrieval-driven multi-path retrieval task, the method of this embodiment automatically adjusts and analyzes the credibility of the results returned by the two retrieval paths, and automatically returns the most reliable and accurate retrieval results. In addition, the above scheme effectively incorporates the advantages of both detection methods and avoids their potential defects through organic combination, effectively reducing missed detections and cold start problems.
[0054] In another embodiment, it may be necessary to return multiple search results with high similarity. In this case, the search results returned by the two search paths can be merged into a unified `query_material_scores` dictionary output. This dictionary uses the material ID as the key and the score as the value. It should be noted that for materials appearing in both paths, the higher score from both paths can be taken as its final score.
[0055] Furthermore, for non-text inputs such as URLs (material address identifiers) or file types, in this embodiment, they are converted into vector retrieval through visual vector bridging for retrieval.
[0056] Specifically, if the recognition result is a material address identifier, the vector retrieval generated based on visual understanding is configured as follows: S501: Determine whether the material address identifier matches the meta description tag in the database.
[0057] S502: If so, directly retrieve the meta description tag and skip step S503, and obtain the cache file from the cache.
[0058] S503: If not, perform visual understanding and keyframe analysis tasks and obtain the parsing file online.
[0059] Specifically, it invokes visual understanding services to generate structured natural language descriptions for images / videos. Particularly for videos, it can extract keyframes (such as cover frames) for analysis.
[0060] S504: Generate a temporary OSS storage URL for cached files and / or parsed files.
[0061] S505: Performs a multi-path retrieval task based on the temporary OSS storage URL, and returns retrieval results other than the material address identifier.
[0062] It should be explained that after generating the natural language structured representation using steps S501-S504, the process returns to step S40 to perform the dual-path retrieval task. During the return process, relevant returned content containing the original material address identifier needs to be removed to achieve deduplication.
[0063] Furthermore, if the identification result is a logical combination, the parallel hybrid retrieval based on the logical combination input is configured as follows: First, the logical combination is broken down into multiple subtasks that are executed in parallel, and the corresponding sub-search strategies are invoked. Then, according to the sub-search strategies, single-path retrieval tasks and / or multi-path retrieval tasks are executed in parallel, and the retrieval results are returned.
[0064] More specifically, during the execution process, the sub-strategies corresponding to each input are first executed in parallel, such as text vector retrieval, URL visual retrieval, and tag filtering. Then, the material result lists returned by all sub-strategies are aggregated into the same dictionary. Next, for the same material that appears in multiple sub-strategies, its highest score is retained. Finally, the results are returned in descending order of the final score.
[0065] Through the above technical solution, the method of this embodiment can support complex hybrid retrieval scenarios and has strong reliability.
[0066] In a preferred embodiment, to standardize historical tags to support the tag-path-based retrieval described above, this embodiment also introduces a tag vector index co-management mechanism. It should be specifically explained that tags not only exist as classification markers but can also serve as first-class vector citizens. Therefore, each tag, when created, must adhere to the following: First, the label name and label description are concatenated into text, then the text-embedding-v4 model is called to generate a 1024-dimensional vector. This vector is then stored in two locations simultaneously. The first is the PostgreSQL vector_info JSON field to support persistence; the second is the Milvus tag_vectors collection to support fast retrieval.
[0067] It is important to note that when a tag is deleted, the corresponding Milvus vector is synchronously cleared through a batch deletion expression (id in [...] id in [...]), thereby maintaining dual-storage consistency.
[0068] Based on the above technical description, and differing from existing technologies, the method of this embodiment has at least the following advantages: Firstly, the method in this embodiment independently searches in both the material vector space and the tag vector space in a single query, and then merges the two results through a material tag bridging table to remove duplicates. This effectively solves the problem of missed detection caused by "different terms in the material description and the query term but matching tags", and effectively improves the retrieval recall rate.
[0069] Secondly, the method in this embodiment can automatically detect which input modalities the user provides (e.g., text keywords, tag ID lists, material URLs, or uploaded files), and automatically select the optimal search strategy based on the input combination, without requiring the user to manually select the search mode. When multiple inputs exist, a fusion strategy is automatically executed to merge the results from multiple sources.
[0070] Third, the method in this embodiment proposes a weighted scoring formula based on the tag hit ratio, which enables materials that match more user-selected tags to obtain higher comprehensive scores, and is superior to materials that match only a few tags but have high single similarity. The search ranking results obtained in this way are more in line with the user's search intent and effectively improve the user experience.
[0071] Fourth, for non-text inputs such as images, video URLs, or file uploads, the method in this embodiment uses AI visual understanding to generate natural language descriptions, text embedding vectorization, and vector retrieval bridging links to achieve semantic retrieval capabilities for non-text inputs, while also avoiding redundant analysis in the database caching layer.
[0072] Example 2 like Figure 2 As shown, this embodiment discloses a multimodal unified material retrieval system based on a dual vector space, including a processor and a memory. The memory stores computer program instructions, and when the computer program instructions are executed by the processor, the multimodal unified material retrieval method based on a dual vector space described in the first aspect is implemented.
[0073] The system in this embodiment also includes other components well known to those skilled in the art, such as communication interfaces. Their settings and functions are known in the art, and therefore will not be described in detail here.
[0074] In this invention, the aforementioned memory can be any tangible medium containing or storing a program that can be used or combined with an instruction execution system, apparatus, or device. For example, a computer-readable storage medium can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc., or any other medium that can be used to store desired information and can be accessed by an application, module, or both. Any such computer storage medium can be part of a device or accessible to or connected to a device. Any application or module described in this invention can be implemented using computer-readable / executable instructions that can be stored or otherwise maintained by such a computer-readable medium.
[0075] In the description of this specification, "multiple" means at least two, such as two, three or more, etc., unless otherwise expressly and specifically defined.
[0076] While this specification has shown and described numerous embodiments of the invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will occur to those skilled in the art without departing from the spirit and essence of the invention. It should be understood that various alternatives to the embodiments of the invention described herein may be employed in the practice of this invention.
Claims
1. A multimodal unified material retrieval method based on a dual vector space, characterized in that, include: Receive multimodal retrieval requests; The multimodal retrieval request is identified to obtain the identification result; Based on the recognition results, a corresponding search strategy is selected; wherein, the search strategy includes at least single vector retrieval and text tag joint vector retrieval; According to the search strategy, a single-path retrieval task or a multi-path retrieval task is executed, and the retrieval results are returned; wherein, if the multi-path retrieval task is executed, the retrieval results returned by the multi-path retrieval task are merged and output.
2. The multimodal unified material retrieval method based on a dual vector space according to claim 1, characterized in that, If the search strategy is the text tag joint vector retrieval, a multi-path retrieval task is performed according to the search strategy, and the retrieval results are returned, including: The multimodal retrieval request is converted into a query vector using a feature transformation model, and a dual-path retrieval parallel thread is started: In the first thread, the query vector is used to perform a first path retrieval task in a preset set of material vectors, and the first retrieval result is returned; In the second thread, the query vector is used to perform a second path retrieval task in a preset tag vector set, and a second retrieval result is returned; The first search result and the second search result are merged and then output.
3. The multimodal unified material retrieval method based on a dual vector space according to claim 2, characterized in that, The first path retrieval task is configured as follows: The query vector is used to perform a cosine similarity search on the material vector set, and the information of several materials with the highest similarity and the material similarity score are returned as the first search result.
4. The multimodal unified material retrieval method based on a dual vector space according to claim 3, characterized in that, The second path retrieval task is configured as follows: The query vector is used to perform a cosine similarity search in the tag vector set, and the top-ranking similar tags are returned. Retrieve the preset material label association table to query the material ID associated with the similar labels; If the same material is matched by multiple similar tags, the highest tag similarity score is taken as the tag path score of the material. The material ID and the tag path score are used as the second search result.
5. The multimodal unified material retrieval method based on a dual vector space according to claim 4, characterized in that, The first search result and the second search result are merged and output, including: Record the number of tags that the material information is matched by the similar tags; Calculate the tag hit ratio weight based on the number of tags; Based on the tag hit ratio weight, the material similarity score and the tag similarity score are weighted and summed to obtain a comprehensive score; The material information with the highest comprehensive score will be returned as the first search result.
6. The multimodal unified material retrieval method based on a dual vector space according to claim 1, characterized in that, The multimodal retrieval request includes, but is not limited to, any one or a logical combination of text keywords, natural query statements, tag ID lists, material address identifiers, images, or videos.
7. The multimodal unified material retrieval method based on a dual vector space according to claim 6, characterized in that, The search strategy also includes vector retrieval based on visual understanding and parallel hybrid retrieval based on logical combination input.
8. The multimodal unified material retrieval method based on a dual vector space according to claim 7, characterized in that, If the recognition result is a material address identifier, the vector retrieval generated based on visual understanding is configured as follows: Determine whether the material address identifier matches the meta description tag in the database; If so, directly retrieve the meta-description tag and skip the visual understanding and keyframe analysis steps, and obtain the cache file from the cache; If not, perform visual understanding and keyframe analysis tasks to obtain the parsing file online; Generate a temporary OSS storage URL for the cached file and / or the parsed file; A multi-path retrieval task is performed based on the temporary OSS storage URL, and the retrieval results are returned except for the material address identifier.
9. The multimodal unified material retrieval method based on a dual vector space according to claim 7, characterized in that, If the identification result is a logical combination, the parallel hybrid retrieval based on the logical combination input is configured as follows: The logic combination is broken down into multiple subtasks that are executed in parallel, and the corresponding sub-search strategies are invoked. Based on the sub-search strategy, single-path retrieval tasks and / or multi-path retrieval tasks are executed in parallel, and retrieval results are returned.
10. A multimodal unified material retrieval system based on a dual vector space, characterized in that, It includes a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the multimodal unified material retrieval method based on a dual vector space as described in any one of claims 1-9 is implemented.
