An accessory identification method and apparatus

Abstract

The application discloses a kind of accessory identification method, the method first obtains accessory image and extracts text information, by inquiring preset accessory electronic catalog, fast matching accessory with identifiable text, improve identification efficiency.When there is no matching first candidate accessory or the number of matches is more than 2, extract image feature vector from accessory image, filter multiple second candidate accessories containing image and text information, cover no text, text recognition invalid or matching not unique scene, solve the problem of narrow application range of single text extraction technology.Secondly, input the accessory image and the second candidate accessory information into the trained visual-language model, and use its graphic-text joint understanding ability to accurately distinguish similar accessories, reduce the mismatch rate, and make up for the defects of insufficient accuracy of single technology.Overall, through progressive multi-modal collaborative identification, the accuracy, consistency and efficiency of accessory identification are significantly improved, ensuring smoothness of accessory transaction and maintenance, and optimizing the user experience.

Description

[0001] This application is a divisional application of the invention application filed on May 27, 2026, with Chinese application number 202610738588.3 and title "A Method and Device for Identifying Accessories". Technical Field

[0002] This invention relates to the field of image processing technology, and in particular to a method and apparatus for identifying accessories. Background Technology

[0003] In the context of parts trading and repair, accurate identification of parts is crucial for ensuring transaction efficiency and repair quality. However, due to the wide variety of parts, significant brand differences, high similarity in appearance, and the lack of markings on some parts, it is difficult for relevant personnel to accurately identify the specific information of the parts. In such cases, manual identification is often used. However, manual identification relies on the experience and professional knowledge of the operators and the quality of the parts images, which has the drawbacks of low identification efficiency, poor consistency, and high misjudgment rate, which seriously affects transaction efficiency and user experience. Therefore, there is an urgent need for a parts identification method. Summary of the Invention

[0004] This invention provides a method and apparatus for identifying spare parts, which can improve the accuracy, consistency and efficiency of spare parts identification, thereby ensuring the smoothness of spare parts transactions and repair processes and improving the user experience.

[0005] In a first aspect, the present invention provides a method for identifying accessories, the method comprising: Acquire images of components and extract text information from those images; The text information is used to search a preset electronic catalog of accessories to obtain the search results; If the number of first candidate accessories matching the accessory image is 0, equal to or greater than 2, then image feature extraction is performed on the accessory image to obtain the image feature vector of the accessory image; Using the image feature vector of the accessory image, accessory information of multiple second candidate accessories that match the image feature vector of the accessory image is selected; wherein, the accessory information of the second candidate accessories includes image information and text information of the second candidate accessories; The accessory image and accessory information of the plurality of second candidate accessories are input into a trained vision-language model to obtain the target candidate accessory corresponding to the accessory image; wherein, the target candidate accessory is one of the plurality of second candidate accessories.

[0006] Secondly, the present invention provides an accessory identification device, the device comprising: Information acquisition unit, used to acquire accessory images and extract text information from the accessory images; The parts query unit is used to query a preset electronic parts catalog using the text information and obtain query results; The feature extraction unit is used to extract image features from the accessory image if the number of first candidate accessories matching the accessory image is 0, equal to or greater than 2, and thus obtain the image feature vector of the accessory image. An information filtering unit is used to filter out accessory information of a plurality of second candidate accessories that match the image feature vector of the accessory image using the image feature vector of the accessory image; wherein, the accessory information of the second candidate accessories includes image information and text information of the second candidate accessories; The accessory determination unit is used to input the accessory image and accessory information of the plurality of second candidate accessories into a trained vision-language model to obtain the target candidate accessory corresponding to the accessory image; wherein, the target candidate accessory is one of the plurality of second candidate accessories.

[0007] Thirdly, the present invention provides a readable medium including executable instructions, which, when executed by a processor of an electronic device, cause the electronic device to perform any of the methods described in the first aspect.

[0008] Fourthly, the present invention provides an electronic device including a processor and a memory storing execution instructions, wherein when the processor executes the execution instructions stored in the memory, the processor performs the method as described in any of the first aspects.

[0009] As can be seen from the above technical solution, the method provided by the present invention first acquires the image of the accessory and extracts the text information therein, and then uses this text information to query a preset electronic catalog of accessories, which can quickly match accessories with identifiable text information, thus improving recognition efficiency; when the query result is that there is no first candidate accessory matching the accessory image or the number of matching first candidate accessories is equal to or greater than 2. In this process, image feature vectors are extracted from the parts images. These vectors are then used to filter out multiple second-candidate parts containing both image and text information. This effectively covers scenarios where there is no text, text recognition is invalid, or the matching results are not unique, thus solving the problem of the narrow applicability of single text extraction technology. By inputting the parts images and the parts information of multiple second-candidate parts into a trained visual-language model, target candidate parts are obtained. The image-text joint understanding capability of the visual-language model is used to accurately distinguish similar parts, reducing the false matching rate of similar parts and making up for the shortcomings of single image feature matching technology and single visual-language model recognition accuracy. The entire process, through progressive processing of text query, image feature filtering, and visual-language model verification, forms a multimodal collaborative recognition mechanism, which significantly improves the accuracy, consistency, and efficiency of parts recognition, thereby ensuring the smoothness of parts trading and repair processes and improving the user experience.

[0010] The further effects of the aforementioned non-conventional preferred method will be explained below in conjunction with specific embodiments. Attached Figure Description

[0011] To more clearly illustrate the embodiments of the present invention or the existing technical solutions, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart illustrating an accessory identification method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of an accessory identification device according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0014] Various non-limiting embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0015] See Figure 1 This illustration shows a component identification method according to an embodiment of the present invention. In this embodiment, the method may include, for example, the following steps: S101: Obtain the accessory image and extract the text information from the accessory image.

[0016] In this embodiment, the accessory image refers to the original image of the automotive accessory to be identified, obtained by the user through a camera. This image clearly shows the accessory's appearance, any inscriptions, and other information, serving as the foundational data for subsequent identification processes. The text information refers to the identification information related to the accessory's identity extracted from the accessory image. The core of this information is the part number in a "number-letter" combination format (e.g., "03523-ABC321"). This information is crucial for quickly matching accessories.

[0017] In this embodiment, after obtaining the accessory image, the text information in the accessory image can be extracted first. As an example, an OCR module can be used to perform text recognition on the accessory image to obtain the text recognition result. Then, the accessory identification information in the text recognition result can be used as the text information in the accessory image. For example, text information in the text recognition result that conforms to a preset accessory identification text format can be used as the text information in the accessory image.

[0018] For example, firstly, the user takes a picture of the car part to be identified using a terminal device (such as a mobile phone, camera, etc.), obtains a clear image of the part, and then inputs it into the part identification system of this invention. The system calls the OCR module (Optical Character Recognition module) to perform text recognition on the input part image: the OCR module first performs scanning preprocessing on the part image to enhance the contrast of the text area in the image, and then extracts all visible text in the image through character detection and recognition algorithms to obtain the text recognition result. The text recognition result is filtered, and the part identification information (i.e., part number, such as "03523-ABC321") that conforms to the "number-letter" combination format is taken as the text information corresponding to the part image; if there is no information that conforms to the above format in the text recognition result, it is determined that the part image has no valid text information.

[0019] S102: Use the text information to query the preset electronic catalog of accessories and obtain the query results.

[0020] In this embodiment, the electronic parts catalog can be understood as a pre-built and stored auto parts-specific data catalog that supports local storage and cloud synchronization. The catalog contains multiple preset parts identification information (i.e., part numbers) and corresponding parts attribute information (such as parts name, brand model, compatible vehicle models, etc.) to enable quick search and matching of parts.

[0021] In this embodiment, the text information can be used to query a preset electronic catalog of parts to obtain query results. As an example, the text information in the part image can be used as a query keyword to query the pre-stored electronic catalog of parts, obtaining query results; that is, querying whether there is a part identifier in the pre-stored electronic catalog that matches the part identifier information in the text recognition result. The electronic catalog of parts stores multiple preset part identifier information and the corresponding part attribute information.

[0022] If the query result is a first candidate accessory that matches the accessory image, then the first candidate accessory is taken as the target candidate accessory corresponding to the accessory image. Here, the first candidate accessory refers to the accessory that may match the accessory image to be identified, obtained by querying the electronic accessory catalog through text information; its number is determined by the query result.

[0023] For example, the text information (part number) extracted from S101 can be used as a query keyword to initiate a query operation on a preset electronic parts catalog. During the query, the locally stored electronic parts catalog is accessed first. If no corresponding query results are found in the local catalog or the query fails, the cloud-based electronic parts catalog is automatically queried to ensure that the query covers all available data. After receiving the query keyword, the electronic parts catalog retrieves the part identification information stored in the catalog using an exact matching algorithm to obtain the query results. The query results include three cases: "No first candidate parts matching the part image (i.e., 0)," "The number of first candidate parts matching the part image is 1," and "The number of first candidate parts matching the part image is equal to or greater than 2."

[0024] If the query result is "the number of first candidate parts matching the part image is 1", then the first candidate part is directly used as the target candidate part corresponding to the part image and the result is output; if the query result is "no first candidate parts matching the part image (i.e. 0)" or "the number of first candidate parts matching the part image is equal to or greater than 2", then proceed to the next step of the image feature extraction process, i.e., S103.

[0025] S103: If the number of first candidate accessories matching the accessory image is 0, equal to or greater than 2, then the accessory image is subjected to image feature extraction to obtain the image feature vector of the accessory image.

[0026] In this embodiment, the image feature vector, also known as the embedding vector, is a fixed-dimensional numerical vector obtained after feature extraction from the accessory image. It can quantitatively represent the local and global features of the accessory image and is the core data for achieving image similarity matching.

[0027] In this embodiment, image features can be extracted from the accessory image to obtain an image feature vector. As an example, the accessory image can first be preprocessed using an image encoder to obtain a processed accessory image; wherein the preprocessing includes at least one of image cropping, size normalization, and pixel normalization. Then, the convolutional and pooling layers in the image encoder can be used to extract local and global features from the processed accessory image. Finally, the output layer of the image encoder can be used to obtain a fixed-dimensional embedding vector based on the local and global features of the processed accessory image, and this fixed-dimensional embedding vector can be used as the image feature vector of the accessory image.

[0028] It should be noted that this embodiment can call an image encoder based on the EfficientNet architecture to perform preprocessing operations on the accessory image. Preprocessing includes image cropping (removing irrelevant background areas from the image while retaining the main body of the accessory), size normalization (adjusting the image to a uniform size, such as 224×224 pixels), and pixel standardization (mapping pixel values ​​to a preset range to eliminate the influence of brightness and contrast differences), resulting in a processed accessory image. The processed accessory image is input into the convolutional layer of the image encoder, where multi-layer convolutional operations extract local features of the image (such as details like the corners, textures, and holes of the accessory). These local features are then input into a pooling layer, where downsampling operations preserve key features and reduce data dimensionality, resulting in global features of the accessory image (such as the overall shape and structural proportions of the accessory). After processing by the output layer of the image encoder, the global features generate a fixed-dimensional (such as 512-dimensional or 1024-dimensional) embedding vector. This embedding vector is the image feature vector of the accessory image, used for subsequent similarity matching.

[0029] S104: Using the image feature vector of the accessory image, filter out the accessory information of multiple second candidate accessories that match the image feature vector of the accessory image.

[0030] The accessory information of the second candidate accessory includes image information and text information of the second candidate accessory. The second candidate accessory refers to multiple accessories with high similarity to the accessory image to be identified, selected by similarity matching of image feature vectors with vectors in a preset vector library. The accessory information includes image information (such as the original image of the second candidate accessory) and text information (such as the accessory name, brand model, appearance parameters, etc. of the second candidate accessory).

[0031] In this embodiment, the image feature vector of the accessory image can be used to filter out accessory information for multiple second candidate accessories that match the image feature vector of the accessory image. As an example, a cosine similarity algorithm can be used to calculate the similarity value between the image feature vector of the accessory image and the feature vectors of each accessory in a preset vector library. The higher the similarity value between the image feature vector of the accessory image and the feature vectors of accessories in the preset vector library, the higher the probability that the accessory image and the accessory feature vector correspond to the same accessory. Conversely, the lower the similarity value between the image feature vector of the accessory image and the feature vectors of accessories in the preset vector library, the lower the probability that the accessory image and the accessory feature vector correspond to the same accessory. The preset vector library stores multiple accessory feature vectors and corresponding accessory information. The accessory information includes image information and text information, and the text information includes at least one of accessory name, brand model, and appearance parameters. Understandably, the pre-defined vector library can be a database using a distributed storage architecture, pre-stored with a massive amount of automotive parts feature vectors (i.e., image feature vectors) and corresponding part information (image information and text information), providing data support for image feature matching.

[0032] Then, the similarity values ​​of the feature vectors of each accessory in the vector library can be sorted from high to low, and the accessory information of the top N accessory feature vectors can be selected as the accessory information of the second candidate accessory, where N is a preset positive integer, for example, N can be 3.

[0033] For example, the system calls the cosine similarity algorithm to calculate the similarity value between the image feature vector of the obtained accessory image and the feature vectors of each accessory in the preset vector library. The cosine similarity algorithm represents the similarity of vectors by calculating the cosine value of the angle between two vectors. The closer the cosine value is to 1, the higher the similarity between the accessories corresponding to the two vectors; the closer the cosine value is to 0, the lower the similarity.

[0034] After the calculation is completed, all accessory feature vectors in the preset vector library are sorted from high to low according to their corresponding similarity values, and the top N accessory feature vectors (N is a configurable positive integer from 3 to 10, and N is preferably 5 in this embodiment) are selected.

[0035] Extract the accessory information corresponding to these N accessory feature vectors, and use it as accessory information for multiple second candidate accessories that match the accessory image to be identified. The accessory information of each second candidate accessory includes the image information (original captured image) and text information (accessory name, brand model, appearance parameters, etc.) of the accessory.

[0036] S105: Input the accessory image and the accessory information of the plurality of second candidate accessories into the trained vision-language model to obtain the target candidate accessory corresponding to the accessory image.

[0037] The target candidate accessory is one of the multiple second candidate accessories. The Visual-Language Model (VLM) refers to an artificial intelligence model trained on large-scale image and text data, possessing the ability to jointly understand images and text. It can fuse image features and text features for semantic analysis, achieving accurate matching and verification of accessories. The target candidate accessory refers to the accessory that best matches the image of the accessory to be identified, determined after text query, image feature matching, and semantic verification by the Visual-Language Model; it is the final identification result of this method.

[0038] In this embodiment, the accessory image and the accessory information of the plurality of second candidate accessories are input into a trained visual-language model to obtain the target candidate accessory corresponding to the accessory image.

[0039] Specifically, prompt words can be constructed first. These prompt words include accessory matching text instructions. For example, the prompt words may include the instruction to "identify the accessory type, model, and key features corresponding to the input accessory image, and output the candidate accessory that best matches the input image, referring to the image and text information of the provided N second candidate accessories." Then, the prompt words, the accessory image, and the accessory information of the multiple second candidate accessories are input into a trained visual-language model to obtain the target candidate accessory corresponding to the accessory image. Specifically, the visual-language model extracts visual features from the image information of the accessory image and the accessory information of the multiple second candidate accessories based on the prompt words to obtain a visual feature vector. It also performs semantic encoding on the text information of the accessory image and the text information of the multiple second candidate accessories to obtain a semantic feature vector. The visual-language model uses a cross-modal attention mechanism to fuse the visual feature vector and the semantic feature vector to obtain a multimodal feature vector. Based on the multimodal feature vector, a cosine similarity algorithm or a cross-entropy loss function is used to calculate the matching degree between the accessory image and each second candidate accessory. The higher the matching degree between the accessory image and the second candidate accessory, the higher the probability that the accessory image and the second candidate accessory are the same accessory; conversely, the lower the matching degree, the lower the probability that the accessory image and the second candidate accessory are the same accessory. Finally, the second candidate accessory with the highest matching degree can be selected as the target candidate accessory corresponding to the accessory image.

[0040] For example, firstly, a prompt word is constructed, which contains a clear text instruction for matching accessories. Specifically, it is to "identify the accessory type, model and key features corresponding to the input accessory image, and refer to the image information and text information of the provided N second candidate accessories, and output the candidate accessory that best matches the input image". This prompt word is used to guide the visual-language model to perform accurate semantic analysis.

[0041] The constructed prompts, the images of the accessories to be identified, and the accessory information (image information and text information) of the selected second candidate accessories are input together into the trained visual-language model.

[0042] The visual-language model performs the following processing: Based on the prompt words, visual features are extracted from the input accessory image and the image information of the second candidate accessory to obtain the corresponding visual feature vectors; at the same time, semantic encoding is performed on the text information (if any) in the accessory image and the text information of the second candidate accessory to convert the text information into a semantic feature vector that can be understood by the computer.

[0043] By utilizing a cross-modal attention mechanism, visual feature vectors and semantic feature vectors are deeply fused to uncover the correlation between images and text, resulting in multimodal feature vectors.

[0044] Based on multimodal feature vectors, the cosine similarity algorithm or cross-entropy loss function is used to calculate the matching degree between the image of the accessory to be identified and each second candidate accessory. The higher the matching degree value, the higher the fit between the two.

[0045] The visual-language model outputs the second candidate accessory with the highest matching degree, which is the target candidate accessory corresponding to the image of the accessory to be identified.

[0046] In one implementation of this embodiment, the method may further include: Display the accessory information of the target candidate accessory and the accessory information of the plurality of second candidate accessories to the user; Based on the user's confirmation information or objection feedback information regarding the target candidate accessory information, the visual-language model is iteratively optimized.

[0047] In this embodiment, the system displays the accessory information of the target candidate accessory (including accessory name, brand and model, similarity score, compatible vehicle model, etc.) and the accessory information of multiple second candidate accessories to the user in a combination of text and images, so as to facilitate comparison and reference for the user.

[0048] After viewing the display results, users can provide feedback on the target candidate accessory through their terminal devices to confirm the identification result (acknowledging the identification result) or to raise objections (pointing out errors in the identification result and providing correct accessory information).

[0049] After receiving user feedback, the system iteratively optimizes the visual-language model: if the user provides confirmation, the matching data (part image, text information, image feature vector, target candidate part information) is added to the preset vector library as positive samples; if the user provides objection feedback, the error reason and correct part information are recorded, the corresponding data in the preset vector library is updated, the feature extraction parameters of the image encoder (such as convolutional layer weights, pooling window size, etc.) are adjusted, and the prompt word strategy of the visual-language model is optimized to improve the subsequent recognition accuracy of the model.

[0050] As can be seen from the above technical solution, the method provided by the present invention first acquires the accessory image and extracts the text information therein, and then uses this text information to query a preset electronic catalog of accessories, which can quickly match accessories with identifiable text information, thus improving recognition efficiency. When the query result is that there is no first candidate accessory matching the accessory image or the number of matching first candidate accessories is equal to or greater than 2, image feature extraction is performed on the accessory image to obtain an image feature vector, and then the image feature vector is used to filter out accessory information of multiple second candidate accessories containing both image information and text information. This effectively covers scenarios where there is no text, text recognition is invalid, or the matching result is not unique, thus solving the problem of single text... This addresses the issue of narrow applicability of extraction techniques. By inputting the image of the accessory and accessory information from multiple second-candidate accessories into a trained visual-language model, target candidate accessories are obtained. The image-text joint understanding capability of the visual-language model is used to accurately distinguish similar accessories, reducing the false matching rate of similar accessories and compensating for the shortcomings of single image feature matching technology and single visual-language model recognition accuracy. The entire process, through progressive processing of text query, image feature screening, and visual-language model verification, forms a multimodal collaborative recognition mechanism, significantly improving the accuracy, consistency, and efficiency of accessory recognition, thereby ensuring the smoothness of accessory transactions and repair processes and enhancing the user experience.

[0051] like Figure 2 The image shows a specific embodiment of the accessory identification device described in this application. The device described in this embodiment is a physical device used to perform the method described in the above embodiments. Its technical solution is essentially the same as that of the above embodiments, and the corresponding descriptions in the above embodiments are also applicable to this embodiment. In this embodiment, the device includes: Information acquisition unit 201 is used to acquire accessory images and extract text information from the accessory images; The parts query unit 202 is used to query a preset electronic parts catalog using the text information to obtain query results; The feature extraction unit 203 is used to extract image features from the accessory image if the number of first candidate accessories matching the accessory image is 0, equal to or greater than 2, and the number of first candidate accessories matching the accessory image is 0. The information filtering unit 204 is used to filter out the accessory information of a plurality of second candidate accessories that match the image feature vector of the accessory image using the image feature vector of the accessory image; wherein, the accessory information of the second candidate accessories includes image information and text information of the second candidate accessories; The accessory determination unit 205 is used to input the accessory image and accessory information of the plurality of second candidate accessories into a trained vision-language model to obtain the target candidate accessory corresponding to the accessory image; wherein, the target candidate accessory is one of the plurality of second candidate accessories.

[0052] Optionally, the device further includes a candidate accessory determination unit, which, after the step of querying a preset electronic catalog of accessories using the text information to obtain a query result, if the query result is a first candidate accessory that matches the accessory image, then uses the first candidate accessory as the target candidate accessory corresponding to the accessory image.

[0053] Optionally, extracting text information from the accessory image includes: The OCR module was used to perform text recognition on the accessory image to obtain the text recognition result; The accessory identification information in the text recognition result is used as the text information in the accessory image.

[0054] Optionally, the step of using the text information to query a preset electronic catalog of accessories and obtaining query results includes: Using the text information in the accessory images as search keywords, the pre-stored electronic catalog of accessories is searched to obtain the search results; The electronic catalog of accessories stores multiple preset accessory identification information and accessory attribute information corresponding to the accessory identification information.

[0055] Optionally, the step of extracting image features from the accessory image to obtain the image feature vector of the accessory image includes: The accessory image is preprocessed using an image encoder to obtain a processed accessory image; wherein the preprocessing includes at least one of image cropping, size normalization, and pixel normalization; The local and global features of the processed accessory image are extracted using the convolutional and pooling layers in the image encoder. The output layer of the image encoder obtains a fixed-dimensional embedding vector based on the local and global features of the processed accessory image, and uses the fixed-dimensional embedding vector as the image feature vector of the accessory image.

[0056] Optionally, the step of using the image feature vector of the accessory image to filter out accessory information for multiple second candidate accessories that match the image feature vector of the accessory image includes: The similarity value between the image feature vector of the accessory image and the feature vector of each accessory in the preset vector library is calculated using the cosine similarity algorithm. The preset vector library stores the accessory feature vectors of multiple accessories and the accessory information corresponding to the accessory feature vectors. The accessory information includes image information and text information. The text information includes at least one of accessory name, brand model, and appearance parameters. The similarity values ​​of the feature vectors of each accessory in the vector library are sorted from high to low, and the accessory information of the top N accessory feature vectors are selected as the accessory information of the second candidate accessory, where N is a preset positive integer.

[0057] Optionally, the step of inputting the accessory image and the accessory information of the plurality of second candidate accessories into a trained vision-language model to obtain the target candidate accessory corresponding to the accessory image includes: Construct prompt words, the prompt words including accessory matching text instructions; The prompt word, the accessory image, and the accessory information of the multiple second candidate accessories are input into a trained visual-language model to obtain the target candidate accessory corresponding to the accessory image.

[0058] Optionally, the step of inputting the prompt word, the accessory image, and the accessory information of the plurality of second candidate accessories into a trained vision-language model to obtain the target candidate accessory corresponding to the accessory image includes: The visual-language model extracts visual features from the image information of the accessory image and the accessory information of the plurality of second candidate accessories based on the prompt words, to obtain a visual feature vector; and performs semantic encoding on the text information of the accessory image and the text information of the accessory information of the plurality of second candidate accessories, to obtain a semantic feature vector. The visual-language model uses a cross-modal attention mechanism to fuse the visual feature vector and the semantic feature vector to obtain a multimodal feature vector. Based on the multimodal feature vector, the matching degree between the accessory image and each second candidate accessory is calculated using a cosine similarity algorithm or a cross-entropy loss function. The second candidate accessory with the highest matching degree is selected as the target candidate accessory corresponding to the accessory image.

[0059] Optionally, the apparatus further includes an iterative optimization unit, used for: Display the accessory information of the target candidate accessory and the accessory information of the plurality of second candidate accessories to the user; Based on the user's confirmation information or objection feedback information regarding the target candidate accessory information, the visual-language model is iteratively optimized.

[0060] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include main memory, such as high-speed random-access memory (RAM), or it may also include non-volatile memory, such as at least one disk storage device. Of course, the electronic device may also include other hardware required for other services.

[0061] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0062] Memory is used to store instructions for execution. Specifically, instructions for execution are computer programs that can be executed. Memory can include main memory and non-volatile memory, and it provides the processor with execution instructions and data.

[0063] In one possible implementation, the processor reads the corresponding execution instructions from non-volatile memory into main memory and then executes them. Alternatively, it may obtain the corresponding execution instructions from other devices to form an accessory identification device at the logical level. The processor executes the execution instructions stored in the memory to implement the accessory identification method provided in any embodiment of the present invention through the executed instructions.

[0064] The above is as described in the present invention. Figure 1The accessory identification device provided in the illustrated embodiment executes a method that can be applied to or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0065] The steps of the method disclosed in the embodiments of this invention can be directly manifested as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0066] This invention also proposes a readable medium that stores execution instructions. When the stored execution instructions are executed by the processor of an electronic device, the electronic device can execute the accessory identification method provided in any embodiment of this invention, and specifically execute the method described above for data query.

[0067] The electronic devices described in the foregoing embodiments may be computers.

[0068] Those skilled in the art will understand that embodiments of the present invention can be provided as methods or computer program products. Therefore, the present invention can be implemented in a completely hardware embodiment, a completely software embodiment, or a combination of software and hardware.

[0069] The various embodiments in this invention are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0070] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0071] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for identifying accessories, characterized in that, The method includes: Acquire images of components and extract text information from those images; The text information is used to search a preset electronic catalog of parts and accessories to obtain the search results; If the number of first candidate accessories matching the accessory image is 0, equal to or greater than 2, then image feature extraction is performed on the accessory image to obtain the image feature vector of the accessory image; Using the image feature vector of the accessory image, accessory information of multiple second candidate accessories that match the image feature vector of the accessory image is selected; wherein, the accessory information of the second candidate accessories includes image information and text information of the second candidate accessories; The accessory image and the accessory information of the plurality of second candidate accessories are input into a trained vision-language model to obtain the target candidate accessory corresponding to the accessory image; wherein, the target candidate accessory is one of the plurality of second candidate accessories; The step of inputting the accessory image and the accessory information of the plurality of second candidate accessories into a trained vision-language model to obtain the target candidate accessory corresponding to the accessory image includes: Construct prompt words, the prompt words including accessory matching text instructions; The prompt word, the accessory image, and the accessory information of the multiple second candidate accessories are input into a trained visual-language model to obtain the target candidate accessory corresponding to the accessory image.

2. The method according to claim 1, characterized in that, After the step of using the text information to query a preset electronic catalog of accessories and obtain the query results, the method further includes: If the query result is a first candidate accessory that matches the accessory image, then the first candidate accessory is taken as the target candidate accessory corresponding to the accessory image.

3. The method according to claim 1, characterized in that, The extraction of text information from the accessory image includes: The OCR module was used to perform text recognition on the accessory image to obtain the text recognition result; The accessory identification information in the text recognition result is used as the text information in the accessory image.

4. The method according to claim 1, characterized in that, The process of using the text information to query a preset electronic catalog of accessories and obtaining query results includes: Using the text information in the accessory images as search keywords, the pre-stored electronic catalog of accessories is searched to obtain the search results; The electronic catalog of accessories stores multiple preset accessory identification information and accessory attribute information corresponding to the accessory identification information.

5. The method according to claim 1, characterized in that, The step of extracting image features from the accessory image to obtain the image feature vector of the accessory image includes: The accessory image is preprocessed using an image encoder to obtain a processed accessory image; wherein, the preprocessing includes at least one of image cropping, size normalization, and pixel normalization; The local and global features of the processed accessory image are extracted using the convolutional and pooling layers in the image encoder. The output layer of the image encoder obtains a fixed-dimensional embedding vector based on the local and global features of the processed accessory image, and uses the fixed-dimensional embedding vector as the image feature vector of the accessory image.

6. The method according to claim 1, characterized in that, The step of inputting the prompt word, the accessory image, and accessory information of the plurality of second candidate accessories into a trained vision-language model to obtain the target candidate accessory corresponding to the accessory image includes: The visual-language model extracts visual features from the image information of the accessory image and the accessory information of the plurality of second candidate accessories based on the prompt words, to obtain a visual feature vector; and performs semantic encoding on the text information of the accessory image and the text information of the accessory information of the plurality of second candidate accessories, to obtain a semantic feature vector. The visual-language model uses a cross-modal attention mechanism to fuse the visual feature vector and the semantic feature vector to obtain a multimodal feature vector. Based on the multimodal feature vector, the matching degree between the accessory image and each second candidate accessory is calculated using a cosine similarity algorithm or a cross-entropy loss function. The second candidate accessory with the highest matching degree is selected as the target candidate accessory corresponding to the accessory image.

7. The accessory identification method according to claim 1, characterized in that, The method further includes: Display the accessory information of the target candidate accessory and the accessory information of the plurality of second candidate accessories to the user; Based on the user's confirmation information or objection feedback information regarding the target candidate accessory information, the visual-language model is iteratively optimized.

8. An accessory identification device, characterized in that, The device includes: Information acquisition unit, used to acquire accessory images and extract text information from the accessory images; The parts query unit is used to query a preset electronic parts catalog using the text information and obtain query results; The feature extraction unit is used to extract image features from the accessory image if the number of first candidate accessories matching the accessory image is 0, equal to or greater than 2, and thus obtain the image feature vector of the accessory image. An information filtering unit is used to filter out accessory information of a plurality of second candidate accessories that match the image feature vector of the accessory image using the image feature vector of the accessory image; wherein, the accessory information of the second candidate accessories includes image information and text information of the second candidate accessories; The accessory determination unit is used to input the accessory image and accessory information of the plurality of second candidate accessories into a trained vision-language model to obtain the target candidate accessory corresponding to the accessory image; wherein, the target candidate accessory is one of the plurality of second candidate accessories.

Images