Object recognition method and device, equipment and medium

Abstract

The invention provides an object recognition method and device, equipment and a medium. According to the method, firstly, a candidate object set associated with a target object is obtained, and the matching degree of each candidate object in the candidate object set and the target object is higher than a preset threshold value; next, a subset of candidate objects satisfying a predetermined screening rule is determined from the set of candidate objects. Then, a recognition result corresponding to the target object is determined by applying the feature representation of the subset of candidate objects to an object recognition model. By means of the scheme, the object recognition efficiency and the object recognition result can be effectively improved.

Description

Technical Field

[0001] Exemplary embodiments of the present disclosure generally relate to computer technology, and more particularly, to an object recognition method, apparatus, device, and computer-readable storage medium. Background Art

[0002] Nowadays, with the development of economic globalization and the Internet, people need to use more and more information on the Internet in their lives. However, when searching for content on the Internet, it is usually difficult for users to directly obtain the required information due to the massive amount of data. Therefore, how to identify the information or objects required by users from the massive data has become a concern. Summary of the invention

[0003] In a first aspect of the present disclosure, an object recognition method is provided. In the method, a candidate object set associated with a target object is obtained, and the degree of match between each candidate object in the candidate object set and the target object is higher than a predetermined threshold; a subset of candidate objects that meet a predetermined screening rule is determined from the candidate object set; and a recognition result corresponding to the target object is determined by applying a feature representation of the candidate object subset to an object recognition model, wherein the object recognition model is trained by taking a feature representation of a second reference object set associated with a first reference object as input and taking the degree of match between the second reference object set and the first reference object as output.

[0004] In a second aspect of the present disclosure, an object matching method is provided. In the method, it is determined whether a first target object and a second target object satisfy a predetermined screening rule; and in response to determining that the first target object and the second target object satisfy the predetermined screening rule, the degree of matching between the first target object and the second target object is determined by applying a feature representation of the first target object and a feature representation of the second target object to an object recognition model, wherein the object recognition model is trained by taking a feature representation of a set of second reference objects associated with the first reference object as input and taking a degree of matching between the set of second reference objects and the first reference object as output.

[0005] In a third aspect of the present disclosure, an object recognition device is provided. The device includes: an acquisition module configured to acquire a set of candidate objects associated with a target object, wherein the degree of match between each candidate object in the set of candidate objects and the target object is higher than a predetermined threshold; a first determination module configured to determine a subset of candidate objects that meet a predetermined screening rule from the set of candidate objects; and a second determination module configured to determine a recognition result corresponding to the target object by applying a feature representation of the subset of candidate objects to an object recognition model, wherein the object recognition model is obtained by training by taking a feature representation of a set of second reference objects associated with a first reference object as input and taking the degree of match between the second reference object set and the first reference object as output.

[0006] In a fourth aspect of the present disclosure, an object matching device is provided. The device includes: an object screening module configured to determine whether a first target object and a second target object satisfy a predetermined screening rule; and an object matching module configured to determine the degree of matching between the first target object and the second target object by applying a feature representation of the first target object and a feature representation of the second target object to an object recognition model in response to determining that the first target object and the second target object satisfy the predetermined screening rule, wherein the object recognition model is trained by taking a feature representation of a set of second reference objects associated with the first reference object as input and taking the degree of matching between the set of second reference objects and the first reference object as output.

[0007] In a fifth aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When the instructions are executed by the at least one processing unit, the device executes the method of the first aspect or the second aspect.

[0008] In a sixth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and the computer program can be executed by a processor to implement the method of the first aspect or the second aspect.

[0009] It should be understood that the contents described in this content section are not intended to limit the key features or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent in the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0011] Figure 1 A schematic diagram showing a process performed by an object recognition device according to some embodiments of the present disclosure;

[0012] Figure 2 A schematic diagram showing a process performed by an object matching device according to some embodiments of the present disclosure;

[0013] Figure 3 A schematic diagram showing an example process of an object recognition method according to some embodiments of the present disclosure;

[0014] Figure 4 A schematic diagram showing an example parameter parsing process according to some embodiments of the present disclosure is shown;

[0015] Figure 5 A schematic diagram showing a recall process according to some embodiments of the present disclosure is shown;

[0016] Figure 6 A schematic diagram of a rough sorting process according to some embodiments of the present disclosure is shown;

[0017] Figure 7 A schematic diagram showing a sample set generation process according to some embodiments of the present disclosure is shown;

[0018] Figure 8 A schematic diagram showing an indicator calculation process according to some embodiments of the present disclosure;

[0019] Fig. 9 A schematic diagram showing a sample set update process according to some embodiments of the present disclosure is shown;

[0020] Fig.10 A schematic diagram showing a sample regression process according to some embodiments of the present disclosure is shown;

[0021] Fig.11 A schematic diagram showing a debugging information storage process according to some embodiments of the present disclosure is shown;

[0022] Fig.12 A schematic diagram showing a recognition result storage process according to some embodiments of the present disclosure;

[0023] Fig.13 A schematic diagram showing an example recognition result storage structure according to some embodiments of the present disclosure;

[0024] Fig.14 A schematic diagram showing service merging according to some embodiments of the present disclosure is shown;

[0025] Fig.15 A flowchart of an object recognition method according to some embodiments of the present disclosure is shown;

[0026] Fig.16 A flowchart of an object matching method according to some embodiments of the present disclosure is shown;

[0027] Fig.17 A schematic structural block diagram of an object recognition device according to some embodiments of the present disclosure is shown;

[0028] Fig.18 A schematic structural block diagram of an object matching device according to some embodiments of the present disclosure is shown; and

[0029] Fig.19 A block diagram of a device capable of implementing various embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0030] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0031] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". The following may also include other explicit and implicit definitions. As used herein, the term "model" can represent the association relationship between various data. For example, the above-mentioned association relationship can be obtained based on a variety of technical solutions currently known and / or to be developed in the future.

[0032] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0033] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0034] For example, in response to receiving the user's input, a prompt message is sent to the user to clearly prompt the user that the operation performed in response to the user's input will require obtaining and using the user's personal information. Thus, the user can independently choose whether to provide personal information to the electronic device, application, server, storage medium or other software or hardware that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0035] As an optional but non-limiting embodiment, in response to receiving the user's input, the prompt information is sent to the user in a manner such as a pop-up window, and the pop-up window may present the prompt information in text form. In addition, the pop-up window may also carry a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0036] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation to the embodiments of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the embodiments of the present disclosure.

[0037] The term "in response to" as used herein refers to a state in which a corresponding event occurs or a condition is satisfied. It will be understood that the timing of executing a subsequent action executed in response to the event or condition is not necessarily strongly related to the time when the event occurs or the condition is satisfied. For example, in some cases, the subsequent action may be executed immediately when the event occurs or the condition is satisfied; while in other cases, the subsequent action may be executed some time after the event occurs or the condition is satisfied.

[0038] The term "Stock Keeping Unit" (SKU) used here is the smallest sellable unit under the product specification. It is a unique identifier assigned to each product or material in the inventory. Usually, SKU consists of numbers, letters or symbols. The inventory, price and other related quantities of the product are related to the SKU. SKU can contain various information about the product, such as brand, model, color, size, packaging, etc.

[0039] The term "standard product unit" (SPU) used herein is a commodity sales unit between a category and a commodity, used to describe the standardized attributes and characteristics of a product. The SPU describes the characteristics and specifications of a product through a set of standardized attributes.

[0040] The term "positive indexing" used here is a form of indexing technology. Positive indexing uses the document identifier as a keyword and records the location information of each word or term in the document through an index table. This indexing method has a relatively simple structure when it is established, and the establishment process is convenient and easy to maintain.

[0041] The term "inverted index" used here is a form of indexing technology. Inverted indexing uses words or phrases as keywords for indexing. The record entry corresponding to the keyword in the table records all documents where the word or phrase appears, as well as the location information of the word or phrase in the document. Inverted indexing can quickly locate all documents containing the query keyword when searching.

[0042] Traditional object recognition schemes include feature extraction, vector retrieval, etc. Object recognition schemes based on feature extraction need to first perform feature extraction to obtain feature representations, such as feature vectors. The extracted features may include color histograms, gray-level co-occurrence matrices (GLCM), histograms of oriented gradients (HOG), etc. Usually, the above features are manually designed and used to describe the key information of the object. Since feature design needs to be done manually, object recognition methods based on feature extraction are usually time-consuming. Moreover, this usually requires professional knowledge and experience, which makes the implementation cost of object recognition methods based on feature extraction high. In addition, manually designed features are often difficult to cover all possible situations, resulting in limited generalization capabilities of the above schemes when faced with new, unseen data.

[0043] Vector-based retrieval methods mainly use vector space models to convert images or objects into high-dimensional vectors, and achieve fast retrieval and recognition by calculating the similarity between vectors. However, although vector-based retrieval methods can achieve fast retrieval and recognition, their performance is limited by the construction of vector space models and the choice of similarity calculation methods. In addition, for object recognition in complex scenes, vector retrieval methods may find it difficult to achieve ideal accuracy.

[0044] In practical applications, the development of object recognition is based on the retrieval architecture. The retrieval architecture originally refers to the system architecture used to match user queries with documents and sort documents in the search engine system. Since then, search engines have not only been applied to the user and document levels, but have also been expanded to applications such as e-commerce, advertising, and life services. Online information retrieval systems, such as vector space models, mainly use keyword-based retrieval, use the bag-of-words model to represent documents and calculate the similarity value with the query. In the scheme of using indexes for search, the class search engine with inverted index as the mainstream framework is applied. In this type of index structure, the engine will pre-process a large amount of document content into a series of objects, such as dictionaries, inverted lists, document information, etc., to build an index library. When receiving a user query, it will match the index library according to the query conditions to obtain a list of document identifiers, and then vectorize it and return the result to the user. The index framework has fast retrieval speed, high accuracy, and supports massive data, but there are problems such as high data preprocessing requirements, large storage, and the effect is limited by the index.

[0045] In order to at least partially solve the above problems and other potential problems, the present disclosure proposes a solution for object recognition. The solution of the present disclosure improves the efficiency of object recognition and the accuracy of recognition results by obtaining a candidate object set associated with a target object, screening a candidate object subset that meets a predetermined screening rule from the candidate object set, and applying a feature representation of the candidate object subset to an object recognition model to determine a recognition result.

[0046] For the convenience of explanation, the following Figure 1 The object recognition process is discussed as an example. Specifically, Figure 1 A schematic diagram of the processing process of the object recognition device 100 according to some embodiments of the present disclosure is shown. The object recognition device 100 can be any type of mobile terminal, fixed terminal or portable terminal, including any device that can use Internet services, such as a mobile phone, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a media computer, a multimedia tablet, a PDA, etc. In addition, the object recognition device can also be implemented as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, and content distribution networks.

[0047] like Figure 1 As shown, the object recognition device 100 can first obtain a target object 102, which can be an object to be recognized input by a user or obtained in any other form. The target object 102 can include any form of content such as pictures and text. As an example, the object can be a commodity. In this case, the example process 100 can be implemented as a process of identifying and retrieving according to the target commodity to obtain an object recognition result associated with the target commodity.

[0048] In addition, after the object recognition device 100 obtains the target object 102, it can also pre-process the target object 102 to obtain a feature representation of the target object. Specifically, for example, when the target object 102 is a specific commodity, the object recognition device can extract a feature vector of the commodity based on the text and image information such as the title and attributes of the commodity. The feature representation can include, but is not limited to, text features, image features, multimodal features, or cross features.

[0049] Based on the target object 102 , the object recognition device 100 obtains a candidate object set 104 associated with the target object 102 , wherein the matching degree of each candidate object in the candidate object set 104 with the target object is higher than a predetermined threshold.

[0050] In some embodiments, the object recognition device 100 may decide whether to add the candidate object to the candidate object set according to whether the matching degree between the candidate object and the target object is greater than. In this case, when the matching degree between the candidate object and the target object is greater than a preset threshold, the object recognition device 100 adds the candidate object to the candidate object set associated with the target object, otherwise the candidate object is not added to the candidate object set. In some embodiments of the present disclosure, the step performed by the object recognition device 100 in box 104 may be referred to as "recall".

[0051] The candidate object may be an object in a pre-set candidate object library. Alternatively, the candidate object may be retrieved online by the object recognition device 100 on the Internet. In addition, the candidate object may also be a combination of a pre-set candidate object library and a candidate object retrieved by the object recognition device 100 on the Internet. It should be understood that the above is merely an example for candidate objects, which is exemplary and not restrictive, and the scope of the present disclosure is not limited in this respect.

[0052] Additionally, the object recognition device 100 may determine the similarity between the feature representation of the target object and the feature representation of each candidate object in the plurality of candidate objects. Furthermore, the object recognition device 100 may determine a set of candidate objects associated with the target object from the plurality of candidate objects based on the similarity. For example, when the target object is a commodity, the object recognition device 100 may calculate the similarity between the target object and the candidate objects based on their multimodal features, text features, image features, cross features, and other features.

[0053] In some embodiments, the object recognition device 100 may sort the candidate objects according to their similarity to the target object, for example, the candidate objects with higher similarity to the target object 102 are ranked higher. In this case, the object recognition device 100 may select a certain number of candidate objects at the top of the ranking and add them to the candidate object set 104. In addition, the number of candidate objects selected into the candidate object set 104 may be preset or dynamically adjusted according to the candidate objects and the target object.

[0054] The object recognition device 100 determines a candidate object subset 106 that satisfies a predetermined screening rule from the candidate object set 104. The predetermined screening rule may be associated with an attribute of the target object 102. In some embodiments, the target object may be, for example, a specific commodity, in which case the object recognition device 100 may further screen the candidate objects in the candidate object set 104 according to brand or category. For example, if the candidate object has the same brand and category as the target object, the object recognition device 100 may select the candidate object to be added to the candidate object subset 106. If the candidate object has a different brand or category from the target object 102, the object recognition device 100 will not add the candidate object to the candidate object subset 106.

[0055] In some implementations, the object recognition device 100 may sort the candidate objects in the candidate object set 104 based on the degree of matching between each object in the candidate object set 104 and the target object. In addition, the object recognition device 100 may select candidate objects that meet a predetermined screening rule from the sorting result as the candidate object subset 106. Specifically, the object recognition device 100 may sort and screen the candidate objects in the candidate object set 104. The sorting may be based on a text similarity algorithm or a picture similarity algorithm, etc., and the embodiments of the present disclosure do not impose any restrictions on this.

[0056] For example, in some embodiments, the object recognition device 100 obtains the degree of match between each candidate object in the candidate object set 104 and the target object 102 based on the text similarity algorithm, and sorts the candidate objects according to the degree of match. In addition, the object recognition device 100 may further select a portion of sequential candidate objects that meet a predetermined screening rule from the sorted candidate objects as the candidate object subset 106. For example, for the candidate object set 104 sorted from large to small in terms of the degree of match with the target object, the object recognition device 100 may select a certain number of candidate objects that meet the predetermined screening rule and are in the top order as the candidate object subset 106.

[0057] In this way, the object recognition device 100 can effectively reduce the number of candidate objects while ensuring the accuracy of the screening results, thereby effectively reducing the complexity of the object recognition process and improving the recognition efficiency of the object recognition process. In addition, the number of candidate objects selected into the candidate object subset 106 can be preset or dynamically adjusted according to the matching degree between the candidate objects and the target object 105.

[0058] After obtaining the candidate object subset 106, the object recognition device 100 may apply the feature representation of the candidate object subset 106 to the object recognition model 108. The object recognition model 108 is obtained by, for example, training by taking the feature representation of the second reference object set associated with the first reference object as input and taking the matching degree between the second reference object set and the first reference object as output.

[0059] Specifically, in the above application process, first, the object recognition device 100 obtains the feature representation of the candidate object in the candidate object subset 106, and the feature representation may include a feature vector. Then, the object recognition device 100 may use the feature vector of the candidate object and the feature vector of the target object as inputs to the object recognition model 108, thereby determining the degree of matching between the candidate object and the target object 102. In some embodiments, the object recognition model 108 may be used to determine the degree of matching between multiple input feature representations.

[0060] In some specific implementations, the object recognition model 108 may include, but is not limited to, a machine learning model based on cosine similarity or extreme gradient boosting (XBG). During the training process of the object recognition model 108, the second reference object set may include a plurality of second reference objects associated with the first reference object. The construction process of the second reference object set may refer to the construction process of the candidate object subset 106 described above.

[0061] In some embodiments, the object recognition device 100 may apply the feature representation of the candidate object subset 106 to the object recognition model 108. Thus, the matching degree between each candidate object in the candidate object subset 106 and the target object 102 may be obtained.

[0062] In addition, the object recognition device 100 can determine a candidate object list from the candidate object subset 106 based on the degree of matching. The candidate objects in the candidate object list match the target object and can be sorted according to a predetermined sorting rule. Specifically, the object recognition device 100 can sort the candidate objects in the candidate object subset 106 according to the degree of matching between them and the target object, and select a certain number of candidate objects with a higher degree of matching to add to the candidate object list. In addition, for the candidate objects in the candidate object list, the object recognition device 100 can sort them according to a predetermined sorting rule. In some implementations, the object recognition device 100 can score the candidate objects in the candidate object list according to the predetermined sorting rule. For example, when the candidate object is a candidate commodity, the object recognition device 100 can score the candidate commodity according to indicators such as commodity price and merchant reputation, and sort the candidate objects in the candidate object list according to the scoring results.

[0063] In this way, the object recognition device 100 can effectively reduce the number of candidate objects while ensuring the accuracy of the screening results, thereby effectively reducing the complexity of the processing process of the object recognition device 100 and improving the efficiency of object recognition.

[0064] Thereafter, the object recognition device 100 may determine a recognition result 110 corresponding to the target object 102. Specifically, the object recognition device 100 may screen the candidate objects in the candidate object subset 106 output by the object recognition model 108 according to the degree of match between the target object 102 and the candidate objects, and select the recognition result corresponding to the target object 102 from the candidate objects. For example, the recognition result 110 may include one or more candidate objects with the highest degree of match with the target object. In addition, the multiple candidate objects in the recognition result 110 may be sorted according to the degree of match between the candidate objects and the target object 102 or other predetermined rules. In some embodiments, the recognition result 110 may include the candidate object list discussed above.

[0065] As discussed above, in some embodiments, the processing of the object recognition device 100 may be initiated by a user, for example, the user inputs a target object 102 (e.g., the name of a specific commodity) to the object recognition device 100. In this case, after obtaining the recognition result 110 for the target object 102, the object recognition device 100 may provide the recognition result 110 to the user. Since the object recognition device 100 according to the present disclosure can accurately obtain candidate objects with a high degree of matching with the target object, the accuracy of the object recognition result is improved, and the user experience can be improved.

[0066] In addition to determining the recognition result for the target object 102, the object recognition device 100 can also determine a regression index associated with the target object. The regression index can be determined, for example, based on a candidate object set, a candidate object subset and a recognition result and a predetermined sample set. The regression index can indicate the recognition effect of the object recognition device 100 on the target object 102. In other words, the regression index can be used to evaluate the recognition results of each stage in the processing process of the object recognition device 100. For example, the regression index determined based on the candidate object set and the predetermined sample set can be used to evaluate the object recognition effect of the candidate object set relative to the predetermined sample. In addition, the regression index determined based on the candidate object subset and the predetermined sample set can be used to evaluate the object recognition effect of the candidate object subset relative to the predetermined sample set. Additionally, the regression index based on the recognition result and the predetermined sample set can be used to evaluate the object recognition effect of the recognition result relative to the predetermined sample set.

[0067] In this case, the predetermined sample set is obtained by annotating the pre-collected data, and the predetermined sample set includes positive examples and negative examples associated with the target object. Specifically, the pre-collected data may include pre-extracted samples and manually annotated samples. In addition, the predetermined sample set may also include temporarily annotated samples, for example, samples individually annotated according to the target objects that the user has recently retrieved more. Additionally, the positive examples in the predetermined sample set may include annotated candidate objects that should be identified as associated with the target object. Correspondingly, the negative examples may include annotated candidate objects that should not be identified as associated with the target object. In some embodiments, in the process of determining the regression index based on the predetermined sample set, the candidate objects that are correctly identified in the recognition result will make the regression index better. Correspondingly, the candidate objects that are not correctly identified in the recognition result will make the regression index worse.

[0068] In some embodiments, regression indicators include, but are not limited to, the accuracy of the candidate object set relative to the predetermined sample set, the recall rate of the candidate object set relative to the predetermined sample set, the accuracy of the candidate object subset relative to the predetermined sample set, the recall rate of the candidate object subset relative to the predetermined sample set, the accuracy of the recognition result relative to the predetermined sample set, and the recall rate of the recognition result relative to the predetermined sample set.

[0069] Specifically, the accuracy rate may be calculated, for example, as follows:

[0070] Accuracy = (number of correctly identified positive samples + number of correctly identified negative samples) / total number of samples.

[0071] The recall rate can be calculated, for example, as follows:

[0072] The number of samples identified correctly / (the number of positive samples identified correctly + the number of samples incorrectly identified as negative examples).

[0073] The accuracy rate can indicate the recognition ability of all samples in the object recognition process. The higher the accuracy rate, the stronger the recognition ability of the samples in the object recognition process. Similarly, the recall rate can indicate the recognition ability of the object recognition process for positive samples. The higher the recall rate, the stronger the recognition ability of the positive samples in the object recognition process. For example, the regression index of the candidate object set relative to the predetermined sample set is based on the accuracy rate and / or recall rate of the candidate objects in the candidate object set relative to the predetermined sample set. The regression index of the candidate object subset relative to the predetermined sample set is based on the accuracy rate and / or recall rate of the candidate objects in the candidate object set relative to the predetermined sample set. There can be multiple regression indicators of the recognition result relative to the predetermined sample set, such as based on the accuracy rate and / or recall rate of the candidate objects in the candidate object set relative to the predetermined sample set.

[0074] Optionally, the object recognition device 100 can also determine the reason why the target object matches or does not match each candidate object in the candidate object set. For example, for a candidate object in the candidate object set, if the object recognition device 100 determines whether to add the candidate object to the candidate object subset based on a predetermined screening rule during the screening of the candidate object subset, the object recognition device 100 stores the reason. In some embodiments, the object recognition device 100 stores the reason in association with the candidate object. In this way, if the recognition result of the candidate object is confirmed as a recognition error, the reason for the error can be determined in the reason information associated with the candidate object. In addition, the object recognition process can also be adjusted based on the reason information to improve.

[0075] In addition, the object recognition device 100 stores the recognition result in association with the target object. In some embodiments, the target object and the candidate objects associated in the recognition result can be stored in the associated nodes of the tree structure. Specifically, the object recognition device 100 can use the target object as the center point and store one or more candidate objects in the recognition result as connecting line edge relationships. In this way, the storage space occupied by the recognition result is small, and the speed of reading the recognition result is fast, thereby improving the practicality of the object recognition scheme according to the embodiment of the present disclosure.

[0076] In addition, in some scenarios, users may need to identify whether two objects are the same object. In view of such situations, the present disclosure proposes a solution for object matching. The solution of the present disclosure determines whether the first target object and the second target object meet the predetermined screening rules, and applies the feature representation of the first target object and the feature representation of the second target object to the object recognition model to determine the degree of matching between the first target object and the second target object. In this way, the efficiency of object matching and the accuracy of matching results can be improved.

[0077] The following Figure 2 The object matching device 200 is used as an example for discussion. Specifically, Figure 2 A schematic diagram showing a processing procedure performed by the object matching device 200 according to some embodiments of the present disclosure is shown.

[0078] like Figure 2 As shown, the object matching device 200 can obtain a first target object 202 and a second target object 204. The first target object 202 can be an object to be identified input by a user or obtained in any other form. In addition, the target object can include any form of content such as pictures and texts.

[0079] Similarly, the second target object 204 may also be an object to be identified that is input by a user or obtained in any other form. In addition, the second target object 204 may include content in any form, such as pictures and text. It should be understood that the content and form of the first target object 202 and the second target object 204 may be the same or different, and the scope of the present disclosure is not limited here.

[0080] The object matching device 200 determines whether the first target object 202 and the second target object 204 satisfy the predetermined screening rule 206. Specifically, the predetermined screening rule 206 may be associated with the attributes of the first target object 202 and the second target object 204. In some embodiments, the first target object 202 and the second target object 204 are commodities, and the predetermined screening rule 206 may be associated with the brand or category of the commodity. For example, if the brand and category of the first target object 202 and the second target object 204 are the same, the object matching device 200 may determine that the first target object 202 and the second target object 204 satisfy the predetermined screening rule 206.

[0081] In some embodiments, in response to determining that at least one of the first target object 202 and the second target object 204 does not satisfy a predetermined filtering rule, the object identification mark may determine that the first target object 202 and the second target object 204 are not matched. For example, if the first target object 202 and the second target object 204 are of different brands or categories, the object matching device 200 may determine that the first target object 202 and the second target object 204 do not satisfy the predetermined filtering rule.

[0082] In response to determining that the first target object 202 and the second target object 204 satisfy the predetermined screening rule, the object matching device 200 may determine the degree of matching between the first target object 202 and the second target object 204 by applying the feature representation of the first target object 202 and the feature representation of the second target object 204 to the object recognition model 208. The object recognition model 208 may be trained by taking the feature representation of the second reference object set associated with the first reference object as input and taking the degree of matching between the second reference object set and the first reference object as output.

[0083] Before applying to the object recognition model 208, the object matching device 200 may first obtain feature representations of the first target object 202 and the second target object 204, and the feature representations may include, for example, feature vectors, etc. In some specific implementations, the feature representations may include, but are not limited to, text features, image features, multimodal features, or cross features, etc.

[0084] Then, the object matching device 200 can use the feature vectors of the first target object 202 and the second target object 204 as inputs to the object recognition model 208, thereby determining the matching degree 210 of the first target object 202 and the second target object 204. In some embodiments, the object recognition model 208 can be used to determine the matching degree between the input multiple feature representations. The object recognition model 208 can include a machine learning model based on, including but not limited to, cosine similarity or extreme gradient boosting (extreme gradient boosting, XBG).

[0085] The object matching device 200 may determine whether the first target object 202 and the second target object 204 match or do not match through the matching degree 210 output by the object recognition model 208 .

[0086] In some implementations, if it is determined that the matching degree 210 between the first target object 202 and the second target object 204 is greater than or equal to a predetermined threshold, the object matching device 200 may determine that the first target object 202 and the second target object 204 are matched. In addition, if it is determined that the matching degree 210 between the first target object 202 and the second target object 204 is less than a predetermined threshold, the object matching device 200 may determine that the first target object 202 and the second target object 204 are not matched.

[0087] In addition, the object matching device 200 may also determine the reason why the first target object 202 and the second target object 204 are matched or unmatched. In addition, the determined reason may also be stored.

[0088] For example, if the object matching device 200 determines based on the object recognition model that the matching degree between the first target object 202 and the second target object 204 is less than a predetermined threshold, the object matching device 200 will determine that the first target object 202 and the second target object 204 do not match. In addition, assuming that the two objects do not belong to the same source, for example, one object comes from an authorized source and the other object comes from an unauthorized source, the object matching device 200 may determine that the reason for the mismatch is "source inconsistency". At the same time, in some embodiments, the object matching device 200 may also store the reason for the mismatch.

[0089] Similarly, in some embodiments, if the object matching device 200 determines that one of the first target object 202 and the second target object 204 does not satisfy the predetermined filtering rule, it can determine that the first target object 202 and the second target object 204 do not match. In this case, the object matching device 200 can determine that the reason why the first target object 202 and the second target object 204 do not match is that one of the first target object 202 and the second target object 204 does not satisfy the predetermined filtering rule. At the same time, the reason for the mismatch can be stored.

[0090] In this way, when the object matching result does not match the actual situation, the error can be checked through the stored judgment reasons, thereby improving the availability and stability of the object matching device 200 according to the embodiment of the present disclosure.

[0091] Combine the following Figure 3-Figure 14 The process performed by the object recognition device 100 and / or the object matching device 200 according to the embodiment of the present application is described.

[0092] Figure 3 FIG. 3 is a schematic diagram showing an object recognition process 300 according to some embodiments of the present disclosure. Figure 3 In the example process shown, a specific commodity is used as an example to describe the target object. Specifically, in the following description, the target object is referred to as a seed commodity. Figure 3 The process 300 in the embodiment may be executed by the object recognition device 100. It should be understood that although the process 300 is described below by taking the object recognition device 100 as an example, this is merely exemplary and is not intended to limit the embodiments of the present disclosure in any way.

[0093] In block 302, the object recognition device 100 may obtain a seed commodity. The seed commodity may be an object to be recognized that is input by a user or obtained in any other form. In addition, the target object may include any form of content such as a picture or text.

[0094] In block 304, the object recognition device 100 may perform parameter parsing. In the example process 300, the parameters of each module may be pre-set. In some embodiments, the parameters of each module may also be dynamically configured. The object recognition device 100 may pass the parameters to be used by each module to each module through parameter parsing. In addition, the object recognition device 100 may also perform verification based on the object recognition result to determine a parameter combination with a better object recognition effect.

[0095] In block 306, the object recognition device 100 may perform online parameter configuration. Specifically, the object recognition device 100 may configure the parameters of each module through a dynamic configuration center.

[0096] In block 308, the object recognition device 100 may obtain seed product features. The seed product features may be, but are not limited to, text features, image features, multimodal features, or cross-features. Specifically, the seed product features may be feature vectors of the seed product.

[0097] In block 310 , the object recognition device 100 may merge features. Specifically, the feature representations to be used in the process 300 may be merged, such as text features, image features, or multimodal features.

[0098] In box 312, the object recognition device 100 can call the positive row software development kit (SDK) service. Specifically, the object recognition device 100 obtains the feature representation for each module by calling the positive row SDK service. In some embodiments, the positive row SDK may include, but is not limited to: positive row mode, only taking the data stored in the positive row without using the real-time calculation of the feature service; positive row priority mode, giving priority to the data stored in the positive row, using the parallel calculation of the feature service; and real-time calculation mode, not using the stored data, using the feature service for real-time calculation. Specifically, for the seed product features, the object recognition device 100 adopts the positive row priority mode by default. In addition, for the features of the recalled candidate products, the object recognition device 100 adopts the positive row mode by default.

[0099] In block 314, the object recognition device 100 may obtain an operator service. Specifically, the object recognition mode encapsulates the obtained feature representation for object recognition to form a serviceable component or interface.

[0100] In block 316, the object recognition device 100 may call the recall module. In some embodiments, the object recognition device 100 first performs data preprocessing. For example, based on the text and image information of the product itself, such as the title and attributes, a feature vector is obtained. Specifically, the feature vector may be generated by a feature vector reasoning model. The feature vector reasoning model may be obtained by training based on information in a product database.

[0101] In box 318, the object recognition device 100 can call the recall strategy service. Specifically, the object recognition device 100 can determine the similarity between the feature representation of the seed commodity and the feature representation of each candidate object in the whole network commodity center 346. In addition, the object recognition device 100 can determine the candidate commodity set associated with the seed commodity from multiple candidate objects based on the similarity. In some embodiments, the object recognition mode can sort the candidate objects according to the similarity with the target object, for example, the candidate objects with higher similarity to the target object are ranked higher. In this case, the object recognition device 100 can select a certain number of candidate objects that are ranked higher in the ranking and add them to the candidate object set.

[0102] In some implementations, when a seed product is input, the object recognition device 100 can recall product identifiers with high similarity to the seed product vector from a multi-path vector library, including multi-path sources such as multimodal features, text features, image features, and cross features. The object recognition device 100 determines which path to use to select a certain number of products ranked high in similarity and then proceeds to the next step.

[0103] In block 320 , the object recognition device 100 may call the recall underlying service. Specifically, the object recognition device 100 may directly call the candidate commodities in the reverse order library.

[0104] In block 322 , the products in the global product center 346 are stored in the inverted database in an inverted format, wherein the storage format is a key-value pair of a product identifier and a vector.

[0105] In block 332, the feature representations of the candidate commodities in the whole network commodity center 346 are stored in the feature library. Specifically, the feature representations may include multimodal features, text features, image features, or cross features, etc. In addition, the feature representations of the candidate commodities in the feature library 332 may be arranged in a forward or reverse order.

[0106] In block 326 , the object recognition device 100 may obtain a feature representation of the recalled candidate commodity. Specifically, the feature representation may include multimodal features, text features, image features, or cross features.

[0107] In block 328 , the object recognition device 100 may perform feature merging on the feature representations of the recalled candidate products.

[0108] In box 330, the object recognition device 100 can call the positive row SDK service. Specifically, the object recognition device 100 obtains the feature representation for each module by calling the positive row SDK service. In some embodiments, the positive row SDK may include, but is not limited to: positive row mode, only taking the data stored in the positive row, and not using the real-time calculation of the feature service; positive row priority mode, giving priority to the data stored in the positive row, using the parallel calculation of the feature service; and real-time calculation mode, not using the stored data, using the feature service for real-time calculation. Specifically, for the seed product features, the object recognition device 100 can use the positive row priority mode by default. In addition, for the features of the recalled candidate products, the object recognition device 100 can use the positive row mode by default.

[0109] In block 334, the object recognition device 100 may call the rough sorting module to perform the rough sorting step. In some embodiments, the object recognition device 100 may determine a subset of candidate commodities that meet a predetermined screening rule from the recalled candidate commodities. The predetermined screening rule may be associated with the attributes of the seed commodity.

[0110] In box 336, the object recognition device 100 can perform rough sorting pre-processing on the recalled candidate commodities. Specifically, the object recognition device 100 can match the seed commodity and the candidate commodity based on the brand, category, etc. to remove the obviously wrong candidate commodities that do not meet the same rule. In some embodiments, the object recognition device 100 can filter the recalled candidate commodities according to brand or category. For example, when the candidate commodity has the same brand and category as the seed commodity, the object recognition device 100 can add the candidate commodity to the candidate commodity subset, and when the candidate commodity has a different brand or category from the seed commodity, the object recognition device 100 can remove the candidate commodity.

[0111] In box 338, the object recognition device 100 can call the coarse sorting model service. The object recognition device 100 can sort the candidate commodities in the candidate commodity set based on the degree of match between each commodity in the candidate commodity set and the seed commodity. In addition, the object recognition device 100 can select candidate commodities that meet the predetermined screening rules from the sorting results. Specifically, the object recognition device 100 can sort and screen the candidate commodities in the candidate commodity set. The sorting can be based on a text similarity algorithm or a picture similarity algorithm, etc. In some embodiments, for a candidate commodity set sorted from large to small in terms of the degree of match with the seed commodity, the object recognition device 100 can select a certain number of candidate commodities that meet the predetermined screening rules and are in the top order.

[0112] In this way, the processing process of the object recognition device 100 (hereinafter referred to as "object recognition process") can effectively reduce the number of candidate commodities while ensuring the accuracy of the screening results, thereby effectively reducing the complexity of the object recognition process and improving the recognition efficiency of the object recognition process. In some implementations, after the rough sorting stage, the number of candidate commodities in the candidate commodity subset can be significantly reduced relative to the number of commodities in the recall result. For example, the recall result obtained in box 318 may include 6,000 commodities, while the commodities obtained after the rough sorting stage may be about 3,000.

[0113] In box 340, the object recognition process enters the fine sorting stage. The object recognition device 100 can apply the feature representation of the candidate products in the candidate product subset remaining after the rough sorting stage to the object recognition model. Specifically, the object recognition model is trained by taking the feature representation of the second reference object set associated with the first reference object as input and the matching degree between the second reference object set and the first reference object as output. Specifically, the input of the object recognition model can be a set of vector matrices, generally image features, text word features, etc., which are calculated through pre-cleaned normalization and trained network features to output a similarity score.

[0114] In block 342, the object recognition device 100 may call a refined ranking model service. The object recognition device 100 may use the feature vectors of the candidate products in the candidate product subset and the feature vectors of the seed products as inputs to the machine learning model 344 to determine the degree of match between the candidate products and the target object. The machine learning model 344 may be an implementation of an object recognition model.

[0115] In addition, the object recognition device 100 can determine a candidate commodity list from the candidate commodity subset based on the degree of matching, and the candidate commodities in the candidate commodity list match the seed commodity and are sorted according to a predetermined sorting rule. Specifically, the object recognition device 100 can sort the candidate objects in the candidate commodity subset according to the degree of matching between them and the target object, and select a certain number of candidate commodities with a higher degree of matching to add to the candidate commodity list. In some implementations, after the fine sorting link, the number of candidate commodities in the candidate commodity subset can be significantly reduced relative to the number of commodities in the rough sorting result. For example, the rough sorting result obtained in box 338 may include 3,000 commodities, while the commodities obtained after the rough sorting link may be about 500.

[0116] In block 348, the object recognition process enters the rule screening phase. For the candidate commodities in the candidate commodity list, the object recognition device 100 can sort and screen according to predetermined sorting rules, such as business-defined blacklist and whitelist logic, price verification rules, authority and / or diversity ecological strategies, and embedded logic strongly related to the business.

[0117] In block 350, the object recognition device 100 may call the rule SDK service. In some implementations, the object recognition device 100 may score the candidate products according to a predetermined sorting rule. For example, the object recognition device 100 may score the candidate products according to indicators such as product price and merchant reputation, and sort the candidate objects according to the scoring results.

[0118] At block 352 , the object identification device 100 may determine a list of similar products associated with the seed product.

[0119] In this way, the number of candidate commodities can be effectively reduced while ensuring the accuracy of the object recognition result, thereby effectively reducing the complexity of the object recognition method, and accurately obtaining candidate commodities with a high degree of matching with the seed commodity, thereby improving the accuracy of the object recognition result. Therefore, the accuracy and efficiency of the object recognition method are improved.

[0120] Reference below Figure 4 An example parameter parsing process 400 according to some embodiments of the present disclosure is described. It should be understood that the example parameter parsing process 400 may be executed by the object recognition device 100 or by other appropriate devices, and the embodiments of the present disclosure are not limited thereto.

[0121] In block 402, the object recognition device 100 obtains experimental parameters. In block 404, the object recognition device 100 obtains online default parameters. Then, in block 406, the object recognition device 100 determines merge parameters based on the experimental parameters and the online default parameters. In some embodiments, the online default parameters may be pre-set or manually input.

[0122] In addition, if the user does not manually enter the parameters, the online default parameters are used. If the user manually enters the parameters, the manually entered parameters are used.

[0123] In block 408, the object recognition device 100 may obtain characteristic parameters associated with the seed commodity characteristics. In some embodiments, the parameter is a parameter library containing parameters that need to be read corresponding to each submodule. For the seed commodity, the parameters in the parameter library can be used to determine which characteristics need to be obtained. These parameters will be used when obtaining the characteristics of the seed commodity.

[0124] In block 410 , the object identification device 100 may obtain seed commodity characteristics based at least in part on the associated parameters.

[0125] At block 412 , the object identification device 100 may obtain parameters associated with the recall session.

[0126] At block 414 , the object recognition device 100 may perform a recall session based at least in part on the associated parameters.

[0127] At block 416 , the object recognition device 100 may obtain parameters associated with the feature representation of the recalled candidate item.

[0128] In block 418 , the object recognition device 100 may obtain a feature representation of the recalled candidate product based at least in part on the associated parameters.

[0129] In block 420 , the object recognition device 100 may call parameters associated with the rough sorting phase, such as which version of the rough sorting service to use, whether xgb needs to be used to start sorting, whether the service pool needs to be filtered, and parameters for concurrent calls.

[0130] At block 422 , the object identification device 100 may perform the coarse sorting phase based at least in part on parameters associated with the coarse sorting phase.

[0131] At block 424 , the object recognition device 100 may recall parameters associated with the refinement phase.

[0132] At block 426 , the object identification device 100 may perform a refinement phase based at least in part on the parameters associated with the refinement phase.

[0133] At block 428 , the object identification device 100 may recall parameters associated with the rule session.

[0134] At block 430 , the object identification device 100 may execute the rule session based at least in part on the parameters associated with the rule session.

[0135] In block 432 , the object recognition device 100 may call parameters associated with the regression index calculation phase, such as which regression indexes are calculated.

[0136] At block 434 , the object identification device 100 may calculate a regression metric based at least in part on parameters associated with the regression metric calculation phase.

[0137] It should be understood that, although the parameter parsing process 400 is described above as being executed by the object recognition device 100 , at least part of the parameter parsing process 400 may also be executed by other devices such as the object matching device 200 .

[0138] For example, the object matching device 200 may obtain experimental parameters in block 402 , and may also obtain online default parameters in block 404 , and determine the merge parameters based on the experimental parameters and the online default parameters in block 406 .

[0139] In some embodiments, the object matching device 200 may obtain feature parameters associated with the first seed product and the second seed product at block 408, and obtain feature representations, such as feature vectors, associated with the first seed product and the second seed product based at least in part on the feature parameters at block 410. The first seed product and the second seed product may be implementations of the first target object and the second target object.

[0140] Furthermore, the object matching device 200 may call parameters associated with the coarse sorting phase at block 420. Furthermore, at block 422, the object matching device 200 may perform the coarse sorting phase based at least in part on the parameters associated with the coarse sorting phase.

[0141] Additionally, the object matching device 200 may call parameters associated with the fine ranking phase at block 424. The object matching device 200 may perform the fine ranking phase at block 426 based at least in part on the parameters associated with the fine ranking phase.

[0142] Similarly, the object matching device 200 may call parameters associated with the rule link at block 428. The object matching device 200 may execute the rule link at block 430 based at least in part on the parameters associated with the rule link.

[0143] In this way, each step in the process performed by the object recognition device 100 can be flexibly performed based on parameters, thereby improving the flexibility and practicality of the object recognition process.

[0144] Reference below Figure 5 The recall process 500 according to some embodiments of the present disclosure is described. In the recall process 500, the object recognition device 100 may first call a recall strategy service, also called a recall strategy fusion service, in block 510 to select a certain number of sorted candidate commodities and add them to a candidate commodity set.

[0145] In block 520 , the object recognition device 100 may call the recall underlying service. Specifically, the object recognition device 100 may directly call the candidate commodities in the reverse order library.

[0146] In block 530 , the object recognition apparatus 100 may call the inverted index library to add the candidate item determined in block 510 to the candidate item set.

[0147] In some embodiments, the recalled candidate commodities are stored in a cluster. In this case, in block 540, the object recognition device 100 may call the cluster storing the recalled candidate commodities to add the candidate commodities determined in block 510 to the candidate commodity set.

[0148] In this way, the object recognition process can effectively reduce the number of candidate objects in the recall process while ensuring the accuracy of the screening results, thereby effectively reducing the complexity of the object recognition process and improving the recognition efficiency of the object recognition process.

[0149] Reference below Figure 6 A rough sorting process 600 according to some embodiments of the present disclosure is described. In the rough sorting process 600, in block 620, the object recognition device 100 may perform rule filtering on the recalled candidate commodities. Specifically, the object recognition device 100 may pre-process the recalled candidate commodities based on the rule platform 610 to remove some obviously wrong candidate commodities that do not meet the same rule.

[0150] In block 630, the object identification device 100 may call the recalled product SPU and the feature information list of the recalled product. In some embodiments, the SPU of the recalled product and the feature information list are stored in association.

[0151] In box 640, the object recognition device 100 can batch calculate the rough sorting process. Specifically, the object recognition device 100 performs the rough sorting process based on the rough sorting parameters 650. The rough sorting parameters 650 may include, for example, which version of the rough sorting service is selected, whether xgb is required to start sorting, whether the service pool needs to be filtered, and parameters of concurrent calls. In the rough sorting process 640, the object recognition device 100 can sort the candidate commodities in the candidate commodity set based on the degree of matching of each commodity in the candidate commodity set with the seed commodity. In addition, the object recognition device 100 can select candidate commodities that meet the predetermined screening rules from the sorting results. Specifically, the object recognition device 100 can sort and screen the candidate commodities in the candidate commodity set. The sorting can be based on a text similarity algorithm or a picture similarity algorithm. In some embodiments, for a candidate commodity set sorted from large to small in terms of the degree of matching with the seed commodity, the object recognition device 100 can select a certain number of candidate commodities that meet the predetermined screening rules.

[0152] In block 660, the object recognition device 100 may generate a coarse ranking score list associated with the recalled commodity SPUs. The coarse ranking score list may be obtained by the similarity between the recalled commodity and the seed commodity calculated in block 640 or by scoring based on the attributes of the recalled commodity and the seed commodity and predetermined rules.

[0153] It should be understood that, although the rough sorting process 600 is described above as being executed by the object recognition device 100 , at least part of the rough sorting process 600 may also be executed by other devices such as the object matching device 200 .

[0154] For example, the object matching device 200 may perform rule filtering on the recalled candidate products in box 620. Specifically, the object matching device 200 may pre-process the first seed product and the second seed product based on the rule platform 610. For example, if the first seed product and the second seed product obviously do not meet the same rule, the object matching device 200 may determine that the first seed product and the second seed product do not match. If the first seed product and the second seed product obviously meet the same rule, the object matching device 200 may perform subsequent steps.

[0155] In addition, the object matching apparatus 200 may call feature information of the first seed commodity and feature information of the second seed commodity.

[0156] The object matching device 200 may perform a rough sorting process based on the rough sorting parameters 650 in box 640. In some embodiments, if the object matching device 200 may determine the rough sorting result based on the matching degree of the first seed product and the second seed product. For example, if the object matching device 200 determines that the matching degree of the first seed product and the second seed product is lower than a preset threshold or the first seed product and the second seed product do not meet the predetermined screening rule, the object matching device 200 determines that the first seed product and the second seed product do not match. If the object matching device 200 determines that the matching degree of the first seed product and the second seed product is higher than the preset threshold and the first seed product and the second seed product meet the predetermined screening rule, the object matching device 200 may perform subsequent steps.

[0157] In this way, the object recognition process can effectively reduce the number of candidate objects while ensuring the accuracy of the screening results during the rough sorting process, thereby effectively reducing the complexity of the object recognition process and improving the recognition efficiency of the object recognition process.

[0158] Reference below Figure 7 The sample set generation process 700 according to some embodiments of the present disclosure is described. It should be understood that the sample set generation process 700 can be executed by the object recognition device 100 or by other appropriate devices, and the embodiments of the present disclosure are not limited thereto.

[0159] In the sample set generation process 700 , at block 710 , a same-style coverage seed table is generated.

[0160] In block 712, corresponding same-item data may be collected according to the same-item coverage seed table to obtain the same-item coverage.

[0161] In block 714, manual annotation is performed on the same item data in 712 to obtain same item coverage annotation. Specifically, the same item coverage annotation is a manually collected same item data set stored in a storage medium, which includes a portion of same item positive examples and a portion of non-same item negative examples.

[0162] Additionally, at block 702, the test seed commodity is input into the sample identification process described above.

[0163] At block 704, an associated recalled commodity may be obtained based on the test seed commodity.

[0164] In block 706 , the recalled product obtained in block 704 may be manually labeled to obtain a recalled product label.

[0165] In block 708 , a dataset temporarily focused on sampling according to industry objectives may be input as a temporary annotation.

[0166] In block 716, the same item coverage annotations, recalled product annotations and temporary annotations in blocks 714, 706 and 708 may be collected into an annotation result table. In addition, the annotation result table is also processed based on predetermined rules to supplement or update product information of some products.

[0167] At block 718 , the temporary result table may be used as a quasi-regression sample set.

[0168] At block 720, a gold panning schedule may be used to supplement or update commodity information on the quasi-regression sample set. The gold panning schedule includes obtaining commodity attribute information from a database or data cooperation department and comparing it with the quasi-regression sample set, and supplementing or updating the quasi-regression sample set based on the commodity attribute information.

[0169] In block 722, the products that are incomplete in attributes and cannot be replenished by the gold rush due to being offline can be removed.

[0170] In block 724 , the quasi-regression sample set processed in block 722 may be determined as a formal regression sample set.

[0171] In this way, the object recognition method can update the sample set data in a timely manner to improve the data accuracy in the object recognition process. Therefore, the accuracy and effectiveness of the object recognition method are improved.

[0172] Reference below Figure 8 The index calculation process 800 according to some embodiments of the present disclosure is described. It should be understood that the index calculation process 800 can be performed by the object recognition device 100 or by other appropriate devices, and the embodiments of the present disclosure are not limited thereto. For the convenience of description, the index calculation process 800 is described below using the object recognition device 100 as an example.

[0173] In the index calculation process 800, the object recognition device 100 first determines whether the input commodity identification type is SPU or SKU in block 802. If the input commodity identification type is SPU, as shown in block 804, the object recognition device 100 can determine the seed commodity SPU. If the input commodity identification type is SKU, as shown in block 806, the object recognition device 100 can determine the seed commodity SKU.

[0174] In block 808, the object recognition device 100 calls multiple versions of pre-labeled positive and negative example samples. Additionally, the positive and negative example samples include different versions corresponding to the identification type of the seed product. For example, the positive and negative example samples corresponding to the SKU type seed product and the SPU type seed product are also different.

[0175] In block 810, the object recognition device 100 obtains positive and negative example samples corresponding to block 804 or block 806 from the sample set 812. Specifically, if it is determined in block 802 that the input commodity identification type is SKU, the object recognition device 100 selects positive and negative example samples corresponding to the seed commodity of the SKU type from the sample set 812. If it is determined in block 802 that the input commodity identification type is SPU, the object recognition device 100 selects positive and negative example samples corresponding to the seed commodity of the SPU type from the sample set 812.

[0176] In block 814 , the object recognition apparatus 100 removes duplicates from the positive and negative example samples obtained in block 810 .

[0177] In block 816, the object recognition device 100 calculates a recall index based on the positive and negative sample samples that have been deduplicated in block 814. The recall index may include the accuracy and / or recall rate of the recall stage, etc. Specifically, in this case, the accuracy and recall rate may be calculated based on the positive and negative sample samples corresponding to the recalled candidate commodities.

[0178] For example, the accuracy can be calculated as:

[0179] Accuracy = (number of correctly identified positive samples + number of correctly identified negative samples) / total number of samples.

[0180] At the same time, the recall rate can be calculated as:

[0181] The number of samples identified correctly / (the number of positive samples identified correctly + the number of samples incorrectly identified as negative examples).

[0182] In block 818, the object recognition device 100 may calculate a rough ranking index based on the positive and negative examples that have been deduplicated in block 814. The rough ranking index may include, for example, the accuracy and / or recall rate of the rough ranking stage. Specifically, the accuracy may be calculated as follows:

[0183] Accuracy = (number of correctly identified positive samples + number of correctly identified negative samples) / total number of samples.

[0184] At the same time, the recall rate can be calculated as follows:

[0185] The number of samples identified correctly / (the number of positive samples identified correctly + the number of samples incorrectly identified as negative examples).

[0186] The precision and recall in the above case are calculated based on the positive and negative samples corresponding to the roughly ranked candidate products.

[0187] In box 820, the object recognition device 100 calculates the refined ranking index based on the positive and negative samples that have been deduplicated in box 814. The refined ranking index may include the accuracy and recall rate of the refined ranking stage. Specifically, the accuracy rate is calculated as follows: accuracy rate = (number of correctly identified positive samples + number of correctly identified negative samples) / total number of samples. The recall rate is calculated as follows: number of correctly identified samples / (number of correctly identified positive samples + number of samples incorrectly identified as negative samples). In this case, the calculation of the accuracy rate and the recall rate is based on the positive and negative samples corresponding to the refined ranked candidate products.

[0188] In block 822, the object recognition device 100 calculates the rule index based on the positive and negative sample samples that have been deduplicated in block 814. The rule index may include the accuracy and recall rate of the rule stage. The accuracy and recall rate may be calculated based on the positive and negative sample samples corresponding to the candidate products screened by the rule. For example, the accuracy rate may be calculated as follows:

[0189] Accuracy = (number of correctly identified positive samples + number of correctly identified negative samples) / total number of samples.

[0190] The recall rate can be calculated as follows:

[0191] The number of samples identified correctly / (the number of positive samples identified correctly + the number of samples incorrectly identified as negative examples).

[0192] In block 824, the recall index, rough sorting index, fine sorting index or rule index obtained by the above calculation is determined as a regression index. Specifically, which index(es) the regression index includes depends on the parameters associated with the regression index calculation.

[0193] In this way, the object recognition device 100 according to the embodiment of the present disclosure can calculate an accurate regression index to indicate the accuracy of the recognition results of each module in the object recognition method, thereby improving the accuracy and reliability of the object recognition device 100.

[0194] Reference below Fig. 9The sample set updating process 900 according to some embodiments of the present disclosure is described. It should be understood that the sample set updating process 900 can be performed by the object recognition device 100 or by other appropriate devices, and the embodiments of the present disclosure are not limited thereto. For the convenience of description, the sample set updating process 900 is described below by taking the object recognition device 100 as an example.

[0195] In block 902 , the object recognition apparatus 100 may obtain a newly added sample set to which a new sample is added.

[0196] In block 904, the object recognition device 100 may perform gold panning scheduling for the newly added sample set. Specifically, the commodity attribute information is obtained from the database or the data cooperation department and compared with the quasi-regression sample set, and the quasi-regression sample set is supplemented or updated based on the commodity attribute information. For some commodities in the newly added sample set, their commodity information has missing parts relative to the commodity attribute information obtained from the database or the data cooperation department. In the gold panning scheduling process, the object recognition device may update the commodity information of these commodities according to the commodity attribute information obtained from the database or the data cooperation department.

[0197] In block 906, the scheduled commodities with incomplete attributes may be removed. Specifically, after the gold panning scheduling, there may still be commodities with insufficient commodity information in the sample set. These commodities that cannot be supplemented with commodity information are removed from the sample set.

[0198] At block 908 , the sample set processed at block 906 may be determined as a final sample set.

[0199] In block 910 , the object recognition device 100 may schedule a sample set service. Specifically, the object recognition device may perform forward feature storage and reverse feature recall on the final sample set.

[0200] In block 912, the object recognition device 100 may perform feature storage in a positive order for the final sample set. Specifically, the object recognition device stores the samples in the final sample set and their feature representations in a storage medium in a positive order.

[0201] In block 914 , the object recognition apparatus 100 may perform reverse sorting on the final sample set that has been stored in the forward sorting order.

[0202] In this way, the updated sample set improves the reliability of the sample set by supplementing commodity information and eliminating invalid samples, and improves the efficiency of subsequent sample set calls by forward and reverse sorting. Therefore, the recognition efficiency and reliability of the object recognition device 100 are improved.

[0203] Reference below Fig.10The sample regression process 1000 according to some embodiments of the present disclosure is described. It should be understood that the sample regression process 1000 can be performed by the object recognition device 100 or by other appropriate devices, and the embodiments of the present disclosure are not limited thereto. For the convenience of description, the sample regression process 1000 is described below by taking the object recognition device 100 as an example.

[0204] In block 1002, the object identification device 100 may determine associated online default parameters and / or experimental parameters for a sample regression process. In addition, the object identification device 100 also obtains a seed commodity for the sample regression process.

[0205] At block 1004 , the object recognition device 100 may perform recall based on the online default parameters and / or experimental parameters of block 1002 .

[0206] In block 1012 , the object recognition device 100 may store the recalled candidate commodities in reverse order during the recall process.

[0207] In box 1014, the object recognition device 100 can deduplicate the recalled candidate commodities stored in reverse order in box 1012. In addition, deduplication is performed based on the commodity granularity. For example, deduplication corresponding to the commodity type is performed for SKU-type commodities and SPU-type commodities. In addition, the object recognition device 100 can also score and sort the recalled candidate commodities stored in reverse order in box 1012. Specifically, the object recognition device 100 can score and sort the recalled candidate commodities in the manner described above, and retain the candidate commodities with scores higher than the score threshold, and eliminate the candidate commodities with scores lower than the score threshold.

[0208] In block 1006 , the object recognition device 100 may obtain the same result through the steps described above.

[0209] In block 1008 , the object recognition device 100 may determine a sample set index based on the same result.

[0210] In this way, the object recognition device 100 according to the embodiment of the present disclosure can achieve obtaining accurate identical results and updating the sample set index.

[0211] Reference below Fig.11 The debugging information storage process 1100 according to some embodiments of the present disclosure is described. It should be understood that the debugging information storage process 1100 can be executed by the object recognition device 100 or by other appropriate devices, and the embodiments of the present disclosure are not limited thereto. For the convenience of description, the debugging information storage process 1100 is described below by taking the object recognition device 100 as an example.

[0212] In block 1102, the object recognition device 100 may store debugging information during the object recognition process. Specifically, the debugging information is stored in association with each module in the object recognition process. The debugging information may include the basis and detailed reasons for tracing back historical same-item results / why they are the same item or not the same item during the object recognition process. In addition, the debugging information may also include parameters and call records, etc.

[0213] In block 1104, the object recognition device 100 may determine whether to store the debugging information. In some embodiments, the object recognition device 100 determines whether to store the debugging information based on a parameter. For example, the parameter includes a flag parameter indicating whether to store the debugging information, and the object recognition device 100 determines whether to store the debugging information based on the flag parameter.

[0214] If the object recognition device 100 determines to store the debugging information, the object recognition device 100 stores the debugging information of the recall phase at block 1106. In addition, the object recognition device 100 stores the debugging information of the coarse sorting phase at block 1108. Additionally, the object recognition device 100 stores the debugging information of the fine sorting phase at block 1110.

[0215] In block 1112, the object recognition device 100 may store the debugging information of the recall phase, the rough sorting phase, and the fine sorting phase in a storage medium. Specifically, the debugging information is stored in association with each module in the object recognition process.

[0216] It should be understood that although the debugging information storing process 1100 is described above as being executed by the object identifying device 100 , at least part of the debugging information storing process 1100 may also be executed by other devices such as the object matching device 200 .

[0217] For example, the object matching device 200 may store debugging information in block 1102, and the debugging information includes information generated in a processing process (hereinafter referred to as "object matching process") of the object matching device 200. Specifically, the debugging information may include the basis and detailed reasons for tracing the matching result / why the first seed product and the second seed product are the same or different in the object matching process.

[0218] The object matching device 200 may determine whether debugging information needs to be stored at block 1104 .

[0219] If the object matching device 200 determines to store the debugging information, the object matching device 200 stores the debugging information of the coarse sorting stage in block 1108. Additionally, in block 1110, the object matching device 200 stores the debugging information of the fine sorting stage.

[0220] In block 1112 , the object matching device 200 may store debugging information of the coarse sorting phase and the fine sorting phase in a storage medium.

[0221] In this way, in the object recognition process according to the embodiments of the present disclosure, debugging information can be stored and retrieved to update and improve the object recognition process, thereby improving the practicality of the object recognition process.

[0222] Reference below Fig.12 and Fig.13 The recognition result storage process 1200 according to some embodiments of the present disclosure is described. It should be understood that the recognition result storage process 1200 can be performed by the object recognition device 100 or by other appropriate devices, and the embodiments of the present disclosure are not limited thereto. For the convenience of description, the recognition result storage process 1200 is described below by taking the object recognition device 100 as an example.

[0223] In the storage process 1200 , the object recognition device 100 may store the seed product in association with the information of the same product acquired through the object recognition process.

[0224] In block 1210 , the object recognition device 100 may obtain information about a seed product and associated products of the same type.

[0225] In block 1220, the object recognition device 100 may determine whether the object recognition process is a cross-website method. Specifically, if the acquired same product information includes product information acquired from other websites or databases, it is determined that the object recognition process is a cross-website method; if the acquired same product information does not include product information acquired from other websites or databases, it is determined that the object recognition process is not a cross-website method.

[0226] In block 1230, if it is determined that the object recognition process is a cross-website method, the object recognition device 100 can create nodes and relationships. In some embodiments, the same result list can be stored as a multi-point edge relationship, so that when the same seed is queried later, the historical same result can be directly reused (the cache time can be customized) without having to go through the whole process for calculation every time.

[0227] In block 1240, the object recognition device 100 may store the seed product and the same product information in the graph storage database as graph data. Specifically, the seed product is stored in the parent node, the same product is stored in the child node, and there is an edge between the node where the seed product is located and the node where the associated same product is located.

[0228] like Fig.13As shown, seed product 1304 is stored in a node, and same-item product 1302 and same-item product 1306 associated with seed product 1304 are stored in a child node connected to the node where seed product 1304 is located. Similarly, seed product 1308 is stored in a node, and same-item product 1310 and same-item product 1306 associated with seed product 1308 are stored in a child node connected to the node where seed product 1308 is located. In addition, there is an edge between the connected parent and child nodes indicating the same relationship in the same-item recognition result.

[0229] It should be understood that, although the recognition result storage process 1200 is described above as being executed by the object recognition device 100 , at least part of the recognition result storage process 1200 may also be executed by other devices such as the object matching device 200 .

[0230] For example, the object matching device 200 may obtain product information of the first seed product and the second seed product. Then, in block 1220, the object matching device 200 may determine whether the object recognition process is a cross-website method. Next, the object matching device 200 may create nodes and relationships in block 1230. The object matching device 200 may store the information of whether the first seed product and the second seed product are the same product in the form of graph data in a storage medium in block 1240.

[0231] In some embodiments, the object matching device 200 may store the first seed product and the second seed product in two nodes. If the first seed product and the second seed product are the same product, the object matching device 200 may establish an edge between the two nodes indicating the same product relationship.

[0232] In this way, the seed product and the information of the same product are associated and stored in the form of graph data, which improves the efficiency of calling the seed product and the information of the same product and saves computing resources.

[0233] According to the following Fig.14 The service merging process 1400 according to some embodiments of the present disclosure is described. It should be understood that the service merging process 1400 may be executed by the object recognition device 100 or by other appropriate devices, and the embodiments of the present disclosure are not limited thereto.

[0234] In some embodiments, while retaining the original development and iteration model of the business, two or more services with strong mutual calls are merged into one service for release to provide a set of automated solutions in the compilation and online stages.

[0235] In the service merging process 1400, service A and service B are merged into a service package (A+B). In addition, library A associated with service A and library B associated with service B are merged. Additionally, namespace A corresponds to library A of service A, and namespace B corresponds to library B of service B. After the merge, the original call will be directly and automatically converted into a local function call. Specifically, before the merge, calling service A and service B respectively calls main(A) and main(B), and after the merge, main(A+B) can be directly called.

[0236] It should be understood that, although the service merging process 1400 is described above as being executed by the object recognition device 100 , at least part of the service merging process 1400 may also be executed by other devices such as the object matching device 200 .

[0237] In this way, the service function call in the embodiment of the present disclosure eliminates the serialization, deserialization and network overhead introduced by the remote call, reduces the service request latency, and improves the stability of the service call.

[0238] Fig.15 FIG. 1 is a flowchart of an object recognition method 1500 according to some embodiments of the present disclosure. The object recognition method 1500 may be executed by the object recognition device 100 discussed above, for example.

[0239] Method 1500 starts at 1510. At 1510, the object recognition device 100 obtains a set of candidate objects associated with a target object. At 1520, the object recognition device 100 determines a subset of candidate objects that meet a predetermined screening rule from the set of candidate objects. At 1530, the object recognition device 100 determines a recognition result corresponding to the target object by applying a feature representation of the subset of candidate objects to an object recognition model. The object recognition model is trained by taking a feature representation of a set of second reference objects associated with a first reference object as input and taking a matching degree of the second reference object set with the first reference object as output.

[0240] In some embodiments, the object recognition device 100 may obtain a set of candidate objects associated with the target object by determining the similarity between a feature representation of the target object and a feature representation of each of a plurality of candidate objects; and determining a set of candidate objects associated with the target object from the plurality of candidate objects based on the similarity.

[0241] In some embodiments, the object recognition device 100 can determine a subset of candidate objects that meet predetermined filtering rules from the candidate object set in the following manner: that is, sorting the candidate objects in the candidate object set based on the degree of matching between each object in the candidate object set and the target object; and selecting candidate objects that meet the predetermined filtering rules from the sorting results as the candidate object subset.

[0242] In some embodiments, the object recognition device 100 may determine a recognition result corresponding to the target object by applying a feature representation of a candidate object subset to an object recognition model to obtain a degree of match between each candidate object in the candidate object subset and the target object; and based on the degree of match, determining a candidate object list from the candidate object subset, wherein the candidate objects in the candidate object list match the target object and are sorted according to a predetermined sorting rule.

[0243] In some embodiments, in method 1500 , the object recognition device 100 may further determine a regression index associated with the target object based on at least one of the candidate object set, the candidate object subset, and the recognition result, and a predetermined sample set.

[0244] In some embodiments, the regression index may include at least one of the following: the accuracy of the candidate object set relative to the predetermined sample set, the recall rate of the candidate object set relative to the predetermined sample set, the accuracy of the candidate object subset relative to the predetermined sample set, the recall rate of the candidate object subset relative to the predetermined sample set, the accuracy of the recognition result relative to the predetermined sample set, and the recall rate of the recognition result relative to the predetermined sample set.

[0245] In some embodiments, the predetermined sample set may be obtained by annotating pre-collected data, and the predetermined sample set may include positive examples and negative examples associated with the target object.

[0246] In some embodiments, in method 1500 , the object recognition device 100 may further determine the reason why the target object matches or does not match each candidate object in the candidate object set.

[0247] In some embodiments, in method 1500 , the object recognition device 100 may further store the recognition result in association with the target object.

[0248] In some embodiments, the feature representation may include at least one of a text feature, an image feature, a multimodal feature, and a cross feature.

[0249] Fig.16 A flow chart of an object matching method 1600 according to some embodiments of the present disclosure is shown. The method 1600 may be executed by the object matching device 200 or other appropriate devices, for example.

[0250] Method 1600 starts at 1610. At 1610, object matching device 200 determines whether a first target object and a second target object satisfy a predetermined screening rule. At 1620, object matching device 200 determines a degree of matching between the first target object and the second target object by applying a feature representation of the first target object and a feature representation of the second target object to an object recognition model in response to determining that the first target object and the second target object satisfy the predetermined screening rule. The object recognition model is trained by taking a feature representation of a set of second reference objects associated with a first reference object as input and taking a degree of matching between the set of second reference objects and the first reference object as output.

[0251] In some embodiments, the object matching device 200 may determine the degree of matching between the first target object and the second target object in the following manner: in response to determining that the degree of matching between the first target object and the second target object is greater than or equal to a predetermined threshold, determining that the first target object and the second target object are matched; and in response to determining that the degree of matching between the first target object and the second target object is less than a predetermined threshold, determining that the first target object and the second target object are not matched.

[0252] In some embodiments, in method 1600 , the object matching device 200 may further determine that the first target object and the second target object are not matched in response to determining that at least one of the first target object and the second target object does not satisfy a predetermined screening rule.

[0253] In some embodiments, in the method 1600 , the object matching device 200 may further determine the reason why the first target object and the second target object are matched or unmatched.

[0254] In some embodiments, the feature representation may include at least one of a text feature, an image feature, a multimodal feature, and a cross feature.

[0255] Fig.17 A block diagram of an object recognition apparatus 1700 according to some embodiments of the present disclosure is shown. The object recognition apparatus 1700 may be implemented as or included in the above-mentioned object recognition device. Each module / component in the object recognition apparatus 1700 may be implemented by hardware, software, firmware, or any combination thereof.

[0256] As shown in the figure, the object recognition device 1700 includes: an acquisition module 1710, configured to acquire a set of candidate objects associated with the target object, each candidate object in the candidate object set having a matching degree with the target object higher than a predetermined threshold; a first determination module 1720, configured to determine a subset of candidate objects that meet a predetermined screening rule from the candidate object set; and a second determination module 1730, configured to determine a recognition result corresponding to the target object by applying a feature representation of the candidate object subset to an object recognition model, wherein the object recognition model is trained by taking a feature representation of a second reference object set associated with the first reference object as input and taking the matching degree of the second reference object set with the first reference object as output.

[0257] In some exemplary embodiments, the acquisition module 1710 may also be configured to: determine the similarity between the feature representation of the target object and the feature representation of each candidate object in multiple candidate objects; and based on the similarity, determine a set of candidate objects associated with the target object from the multiple candidate objects.

[0258] In some exemplary embodiments, the first determination module 1720 may also be configured to: sort the candidate objects in the candidate object set based on the degree of match between each object in the candidate object set and the target object; and select candidate objects that meet predetermined screening rules from the sorting results as the candidate object subset.

[0259] In some exemplary embodiments, the second determination module 1730 may also be configured to apply the feature representation of the candidate object subset to the object recognition model to obtain the degree of match between each candidate object in the candidate object subset and the target object; and based on the degree of match, determine a candidate object list from the candidate object subset, wherein the candidate objects in the candidate object list match the target object and are sorted according to a predetermined sorting rule.

[0260] In some exemplary embodiments, the object recognition apparatus 1700 may further include: a third determination module configured to determine a regression index associated with the target object based on at least one of the candidate object set, the candidate object subset and the recognition result and a predetermined sample set.

[0261] In some exemplary embodiments, the regression index may include at least one of the following: the accuracy of the candidate object set relative to the predetermined sample set, the recall rate of the candidate object set relative to the predetermined sample set, the accuracy of the candidate object subset relative to the predetermined sample set, the recall rate of the candidate object subset relative to the predetermined sample set, the accuracy of the recognition result relative to the predetermined sample set, and the recall rate of the recognition result relative to the predetermined sample set.

[0262] In some exemplary embodiments, the predetermined sample set may be obtained by annotating pre-collected data, and the predetermined sample set may include positive examples and negative examples associated with the target object.

[0263] In some exemplary embodiments, the object recognition apparatus 1700 may further include: a fourth determination module configured to determine the reason why the target object matches or does not match each candidate object in the candidate object set.

[0264] In some exemplary embodiments, the object recognition apparatus 1700 may further include: a storage module configured to store the recognition result in association with the target object.

[0265] In some exemplary embodiments, the feature representation may include at least one of a text feature, an image feature, a multimodal feature, and a cross feature.

[0266] Fig.18 A block diagram of an object matching apparatus 1800 according to some embodiments of the present disclosure is shown. The object matching apparatus 1800 may be implemented as or included in the above-mentioned object matching device 200. Each module / component in the object matching apparatus 1800 may be implemented by hardware, software, firmware or any combination thereof.

[0267] As shown in the figure, the object matching device 1800 includes: an object screening module 1810, configured to determine whether the first target object and the second target object satisfy a predetermined screening rule; and an object matching module 1820, configured to determine the degree of matching between the first target object and the second target object by applying a feature representation of the first target object and a feature representation of the second target object to an object recognition model in response to determining that the first target object and the second target object satisfy the predetermined screening rule, wherein the object recognition model is trained by taking a feature representation of a set of second reference objects associated with the first reference object as input and taking the degree of matching between the second reference object set and the first reference object as output.

[0268] In some exemplary embodiments, the object matching module 1820 may also be configured to determine that the first target object and the second target object are matched in response to determining that the degree of match between the first target object and the second target object is greater than or equal to a predetermined threshold; and to determine that the first target object and the second target object are not matched in response to determining that the degree of match between the first target object and the second target object is less than a predetermined threshold.

[0269] In some exemplary embodiments, the object screening module 1810 may also be configured to determine that the first target object and the second target object are unmatched in response to determining that at least one of the first target object and the second target object does not satisfy a predetermined screening rule.

[0270] In some exemplary embodiments, the object matching device 1800 may further include: a reason determination module configured to: determine the reason why the first target object and the second target object are matched or unmatched.

[0271] In some exemplary embodiments, the feature representation may include at least one of a text feature, an image feature, a multimodal feature, and a cross feature.

[0272] Fig.19 1 shows a block diagram of an electronic device 1900 capable of implementing an embodiment of the present disclosure. It should be understood that Fig.19 The electronic device 1900 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein. Fig.19 The electronic device 1900 shown can be used to implement the methods described above. Fig.19 The electronic device 1900 shown can be used to implement Figure 1 The object recognition device 100 or Figure 2 The object matching device 200 in the embodiment can also be used to implement Fig.17 The object recognition device 1700 or Fig.18 Object matching device 1800 in.

[0273] like Fig.19 As shown, the electronic device 1900 is in the form of a general-purpose computing device. The components of the electronic device 1900 may include, but are not limited to, one or more processors or processing units 1910, a memory 1920, a storage device 1930, one or more communication units 1940, one or more input devices 1950, and one or more output devices 1960. The processing unit 1910 may be an actual or virtual processor and is capable of performing various processes according to a program stored in the memory 1920. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capabilities of the electronic device 1900.

[0274] The electronic device 1900 typically includes a plurality of computer storage media. Such media may be any available media accessible to the electronic device 1900, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 1920 may be a volatile memory (e.g., a register, a cache, a random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 1930 may be a removable or non-removable medium, and may include a machine-readable medium, such as a flash drive, a disk, or any other medium, which may be capable of being used to store information and / or data (e.g., training data for training) and may be accessed within the electronic device 1900.

[0275] The electronic device 1900 may further include additional removable / non-removable, volatile / non-volatile storage media. Fig.19 As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. Memory 1920 may include a computer program product 1925 having one or more program modules that are configured to perform various methods or actions of various embodiments of the present disclosure.

[0276] The communication unit 1940 implements communication with other computing devices through a communication medium. Additionally, the functions of the components of the electronic device 1900 can be implemented in a single computing cluster or multiple computing machines that can communicate through a communication connection. Therefore, the electronic device 1900 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0277] The input device 1950 may be one or more input devices, such as a mouse, a keyboard, a tracking ball, etc. The output device 1960 may be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 1900 may also communicate with one or more external devices (not shown) through the communication unit 1940 as needed, such as a storage device, a display device, etc., communicate with one or more devices that allow a user to interact with the electronic device 1900, or communicate with any device that allows the electronic device 1900 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0278] According to an exemplary embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary embodiment of the present disclosure, a computer program product is also provided, the computer program product is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary embodiment of the present disclosure, a computer program product is provided, on which a computer program is stored, and when the program is executed by a processor, the method described above is implemented.

[0279] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices, equipment, and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0280] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0281] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0282] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0283] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the marketplace, or to enable other persons of ordinary skill in the art to understand the various embodiments disclosed herein.

Claims

1. A method for object recognition, comprising: Acquire a set of candidate objects associated with a target object, wherein a matching degree between each candidate object in the set of candidate objects and the target object is higher than a predetermined threshold; Determining a subset of candidate objects that meet a predetermined screening rule from the candidate object set; as well as A recognition result corresponding to the target object is determined by applying the feature representation of the candidate object subset to an object recognition model, wherein the object recognition model is trained by taking the feature representation of a second reference object set associated with a first reference object as input and taking the degree of matching between the second reference object set and the first reference object as output.

2. The method according to claim 1, wherein obtaining a set of candidate objects associated with the target object comprises: Determining a similarity between a feature representation of the target object and a feature representation of each of a plurality of candidate objects; as well as Based on the similarity, the candidate object set associated with the target object is determined from the multiple candidate objects.

3. The method according to claim 1, wherein determining a subset of candidate objects satisfying a predetermined screening rule from the set of candidate objects comprises: sorting the candidate objects in the candidate object set based on the degree of matching between each object in the candidate object set and the target object; as well as The candidate objects satisfying the predetermined screening rule are selected from the sorting results as the candidate object subset.

4. The method according to claim 1, wherein determining a recognition result corresponding to the target object comprises: Applying the feature representation of the candidate object subset to the object recognition model to obtain a matching degree between each candidate object in the candidate object subset and the target object; as well as Based on the matching degree, a candidate object list is determined from the candidate object subset, wherein the candidate objects in the candidate object list match the target object and are sorted according to a predetermined sorting rule.

5. The method according to claim 1, further comprising: Based on at least one of the candidate object set, the candidate object subset and the recognition result and a predetermined sample set, a regression index associated with the target object is determined.

6. The method according to claim 5, wherein the regression index comprises at least one of the following: the accuracy of the candidate object set relative to the predetermined sample set, The recall rate of the candidate object set relative to the predetermined sample set, the accuracy of the candidate object subset relative to the predetermined sample set, The recall rate of the candidate object subset relative to the predetermined sample set, the accuracy of the recognition result relative to the predetermined sample set, and The recall rate of the recognition result relative to the predetermined sample set.

7. The method according to claim 5, wherein the predetermined sample set is obtained by annotating pre-collected data, and the predetermined sample set includes positive examples and negative examples associated with the target object.

8. The method according to claim 1, further comprising: A reason is determined as to whether the target object matches or does not match each candidate object in the set of candidate objects.

9. The method according to claim 1, further comprising: The recognition result is stored in association with the target object.

10. The method according to any one of claims 1 to 9, wherein the feature representation comprises at least one of a text feature, an image feature, a multimodal feature, and a cross feature.

11. An object matching method, comprising: Determining whether the first target object and the second target object meet a predetermined screening rule; as well as In response to determining that the first target object and the second target object satisfy a predetermined screening rule, determining a degree of match between the first target object and the second target object by applying a feature representation of the first target object and a feature representation of the second target object to an object recognition model, The object recognition model is trained by taking the feature representation of a second reference object set associated with a first reference object as input and taking the matching degree between the second reference object set and the first reference object as output.

12. The method according to claim 11, wherein determining the degree of match between the first target object and the second target object comprises: In response to determining that the degree of match between the first target object and the second target object is greater than or equal to a predetermined threshold, determining that the first target object and the second target object are matched; as well as In response to determining that the degree of matching between the first target object and the second target object is less than the predetermined threshold, it is determined that the first target object and the second target object are not matched.

13. The method according to claim 11, further comprising: In response to determining that at least one of the first target object and the second target object does not satisfy the predetermined screening rule, it is determined that the first target object and the second target object are unmatched.

14. The method according to claim 11, further comprising: A reason for determining whether the first target object and the second target object match or do not match.

15. The method according to any one of claims 11 to 14, wherein the feature representation comprises at least one of a text feature, an image feature, a multimodal feature, and a cross feature.

16. An object recognition device, comprising: An acquisition module is configured to acquire a set of candidate objects associated with a target object, wherein a matching degree between each candidate object in the set of candidate objects and the target object is higher than a predetermined threshold; A first determination module is configured to determine a subset of candidate objects that meet a predetermined screening rule from the candidate object set; as well as The second determination module is configured to determine a recognition result corresponding to the target object by applying the feature representation of the candidate object subset to an object recognition model, wherein the object recognition model is trained by taking the feature representation of a second reference object set associated with the first reference object as input and taking the degree of matching between the second reference object set and the first reference object as output.

17. An object matching device, comprising: An object screening module is configured to determine whether the first target object and the second target object meet a predetermined screening rule; as well as an object matching module, configured to, in response to determining that the first target object and the second target object satisfy a predetermined screening rule, determine a degree of matching between the first target object and the second target object by applying a feature representation of the first target object and a feature representation of the second target object to an object recognition model, The object recognition model is trained by taking the feature representation of a second reference object set associated with a first reference object as input and taking the matching degree between the second reference object set and the first reference object as output.

18. An electronic device comprising: at least one processing unit; as well as At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the device to perform the method according to any one of claims 1 to 10 or the method according to any one of claims 11 to 15.

19. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 10 or the method according to any one of claims 11 to 15 is implemented.

Images